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INTRODUCTION 

BACKGROUND . 

The  Cockpit  Display  of  Traffic  Information  (CDT1)  is  a  concept  which  assumes  that 
traffic  information  can  be  made  available  in  a  pictorial  display  to  the  pilot/ 
flight  crew.  Potential  benefits  attributable  to  the  CDT1  concept  have  been 
postulated,  including  (1)  pilot  monitoring  of  air  traffic  control  (ATC) , 
(2)  situational  awareness  with  respect  to  other  air  traffic,  and  (3)  pilot- 
assisted  spacing  and  merging.  However,  operational  use  of  a  CDT1  device  may  have 
certain  limitations,  such  as  increased  pilot  workload,  misinterpretation  of 
situations,  and  distraction  from  other  duties. 

PURPOSE. 

One  of  the  objectives  of  tne  General  Aviation  cockpit  studies  of  CDTl  is  to  assess 
pilot  workload  using  objective,  subjective,  and  physiological  measures.  To  this 
end,  this  literature  search  was  conducted  to  enumerate  several  workload  measurement 
techniques  which  may  have  potential  application  in  the  evaluation  of  pilot  workload 
with  the  operational  use  of  a  CDTl. 

SCOPE. 

Specific  topics  covered  in  this  bibliography,  as  they  relate  to  CDTl  and  its 
concomitant  workload  considerations,  are  physiological  measures  of  workload 
including  heart  race,  s inusarry thmi a  and  oculomecer  analysis;  subjective  measures 
of  workload  highlighting  studies  in  which  both  structured  and  unstructured 
questionnaires  and  interviews  were  administered  to  test  pilots  evaluating  their 
environment;  task  analysis;  and  performance  evaluation.  The  topics  of  fatigue  and 
noise  are  touched  upon.  Only  chose  sources  containing  information  which  in  some 
way  relates  to  the  project  are  included.  However,  it  in  hoped  that  the  scope  of 
the  literature  review  is  broad  enough  to  be  of  value  to  other  researcn  efforts  that 
are  investigating  workload. 

The  sources  contained  in  this  bibliography  si e  listed  alphabetically  by  the 
author's  last  name.  A  subject  index  is  also  included  for  the  convenience  of  the 
user . 

Several  documents  which  contain  an  overview  of  workload  assessment  techniques  are 
included.  One  such  article,  entitled  "Pilot  Workload  and  Fatigue:  A  Critical 
Survey  of  Concepts  and  Assessment  Techniques,"  by  Walter  B.  Gartner  and  Miles  R. 
Murphy  contains  tables  which  delineate  workload  and  fatigue  indicators  with  refer¬ 
ences  to  the  literature.  These  have  been  abridged  and  are  included  in  the  appendix 
of  this  document. 

METHODOLOGY ■ 

A  comprehensive  search  of  the  scientific  literature  dealing  with  various  topics 
related  to  workload  assessment  techniques  was  undertaken.  Several  sources  were 
used  for  this  purpose,  including  books,  scientific  journals,  proceedings  of 
relevant  technical  meetings,  ana  technical  reports. 
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Computerized  information  retrieval  and  a  manual  literature  search  were  both 
conducted  to  locate  the  most  relevant  and  current  literature.  Due  to  the  vast 
amount  of  literature  available,  a  self-imposed  limitation  seemed  appropriate;  that 
was,  to  extract  references  from  approximately  ’970  on,  except  in  the  case  of 
exceptional  sources  that  could  not  be  overlooked. 

COMPUTERIZED  INFORMATION  RETRIEVAL.  The  computerized  information  retrieval  system 
used  is  available  at  the  Federal  Aviation  Administration  (FAA)  Technical  Center 
Library.  Several  relevant  sources  were  located  through  the  use  of  the  many 
available  data  bases.  The  specific  ones  accessed  were  the  National  Technical 
Information  Service  (NTIS),  Scisearch,  Inspec,  Smithsonian  Science  Information 
Exchange  (SSIE),  Phy schological  Abstracts  (PSYCH  INFO),  and  Transportation  Research 
Information  Service  (TRIS).  A  brief  description  of  each  follows. 

NTIS .  NTIS  data  base  consists  of  government-sponsored  research,  development, 
and  engineering,  plus  analyses  prepared  by  Federal  agencies,  their  contractors  or 
grantees.  It  is  the  means  by  which  unclassified,  unlimited  reports  are  made 
available  for  sale  from  agencies  like  National  Aeronautics  and  Space  Administration 
(NASA),  Department  of  Health,  Education,  and  Welfare  (HEW),  Housing  and  Urban 
Development  (HUD),  Department  of  Transportation  (DOT),  Department  of  Commerce,  and 
state  and  local  agencies.  This  data  base  includes  material  from  both  hard  and  soft 
sciences . 

SCISEARCH .  Scisearch  is  a  multidisciplinary  index  to  the  literature  of 
science  and  technology  prepared  by  the  Institute  for  Science  Information.  Ninety 
percent  of  the  world's  significant  scientific  and  technical  literature  is  included 
in  Scisearch.  Citation  indexing  is  provided  which  allows  retrieval  of  recently 
published  articles  through  the  subject  relationships  established  by  an  author's 
reference  or  prior  articles.  Scisearch  covers  every  area  of  the  pure  and  applied 
sciences. 

SSIE.  This  data  base  contains  reports  of  both  government  and  privately  funded 
scientific  research  projects,  either  currently  in  progress  or  initiated  and 
completed  during  the  last  2  years.  This  source  provided  information  on  current 
research  long  before  initial  or  followup  reports  appear  in  the  published 
literature.  SSIE  encompasses  all  fields  of  basic  and  applied  research  in  the  life, 
physical,  social,  and  engineering  sciences.  Ninety  percent  of  the  information  in 
the  data  base  is  provided  by  agencies  of  the  Federal  Government. 

INSPEC.  The  Science  Abstracts  (INSPEC)  forms  the  largest  English-language 
data  base  in  the  field  of  physics,  electrotechnology,  computers,  and  control.  The 
on-line  INSPEC  file  corresponds  to  the  printed  physics  abstracts,  electrical  and 
electronics  abstracts,  computer  and  control  aspects,  journal  papers,  conference 
proceedings,  technical  reports  books,  patents,  and  university  theses  are  abstracted 
and  indexed  for  inclusion  in  the  INSPEC  data  bases. 

PSYCH  INFO.  Psychological  Abstracts  covers  the  world's  literature  in 
psychology  and  related  disciplines  in  the  behavioral  sciences.  Original  research, 
reviews,  discussions,  theory,  conference  reports,  panel  discussions,  case  studies, 
and  descriptions  of  apparatus  are  included. 
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TRIS.  Transportation  research  information  in  air,  highway,  rail,  and  maritime 
transport  is  included.  Subjects  included  are  regulations  and  legislation;  energy, 
environmental  and  safety  concerns;  materials,  design,  construction  and  maintenance 
technology;  and  operations,  traffic  control,  and  communications. 

MANUAL  LITERATURE  SEARCH.  A  manual  search  through  the  literature  was  also 
conducted.  The  journals  "Human  Factors"  and  "Ergonomics"  proved  especially 
fruitful,  as  did  the  text  "Mental  Workload:  Its  Theory  and  Measurement"  edited  by 
Neville  Moray.  This  text  offers  a  collection  of  articles  on  current  research  by 
leading  investigators.  To  supplement  the  computer  and  manual  search  efforts, 
existing  bibliographies  on  operator  workload  were  consulted.  One  such  source  was 
entitled  "A  Survey  and  Analysis  of  Operator  Workload  Assessment  Techniques"  by 
Walter  W.  Wierville  and  Robert  C,  Williges.  This  reference,  in  addition  to  con¬ 
tributing  a  thorough  review  of  the  literature,  published  a  listing  of  relevant 
Advisory  Group  for  Aerospace  Research  and  Development  (AGARD)  publications. 

SOURCE  SELECTION.  The  criteria  for  selection  of  sources  included  in  this 
annotation  were  the  following: 

1.  Workload  -  emphasis  on  mental  workload  versus  physical  workload. 

2.  Cockpit  -  (or  pilot)  as  opposed  to  operator  workload  per  se  (i.e., 
driving  a  car)  unless  the  information  appeared  relevant  to  an  understanding  of 
pilot  workload  in  the  cockpit. 

3.  Background  -  articles  providing  background  information  necessary  for  the 
development  of  a  mental  workload  concept  or  definition. 

4.  Subjects  -  only  work  describing  experiments  with  normal  adults,  preferably 
pilots . 

5.  Application  -  documents  applicable  to  aircrew  mental  workload  assessment 
or  cockpit  workload  assessment. 

6.  Type  of  Measure  -  workload  measures  of  interest  — -  physiological, 
subjective,  or  behavioral/objective. 

In  all  cases  a  document,  whether  it  referred  to  a  specific  measurement  technique, 
mathematical  model,  design  methodology,  or  clinical  methodology,  was  included  only 
if  specific  implications  were  made  regarding  mental  workload  concepts. 

RESULTS. 

The  results  of  the  search  efforts  are  included  here  in  the  form  of  an  annotated 
bibliography.  The  articles  were  summarized,  and  every  effort  was  made  to 
adequately  reflect  the  nature  of  the  information  in  the  text  or  article,  and 
relevant  charts  and  diagrams  were  also  included.  Many  authors  discuss  various 
workload  assessment  techniques,  detailing  their  uses,  ease  of  application,  and 
nature  of  equipment  needed.  Excerpts  from  such  summary  tables  have  been  included, 
where  appropriate.  The  authors'  abstracts  are  used  for  three  specific  references. 
These  include  the  articles  of  Jex,  McDonnell,  and  Phatak;  McHugh,  Brictson,  and 
Naitah;  and  Spyker,  Stackhouse,  Khalafalla,  McLane.  The  most  striking  result  of 
the  search  is  the  lack  of  a  clear-cut  definition  of  workload  that  is  generally 
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valid  and  practically  useful  among  workload  inve s t igators .  Therefore,  each 
investigator  had  to  give  his  own  definition,  which  is  mostly  very  practical,  baaed 
on  the  unique  situation  to  be  investigated.  The  annotations  contained  herein 
include  some  situation-specific  definitions  of  workload  which  the  reader  may  find 
helpful . 

SUMMARY  OF  WORKLOAD  MEASURES 


In  this  section,  several  workload  assessment  methods  are  summarized.  Should  the 
reader  desire  more  information  on  any  one  method  or  combination  of  methods,  this 
can  be  obtained  through  a  relevant  source  found  in  the  annotations  listed  in  the 
Bibliography  section  of  this  report.  In  all  cases,  techniques  summarized  refer  to 
the  measurement  of  mental  workload  as  opposed  to  physical  workload. 

SUBJECTIVE  MEASURES. 

The  use  of  subjective  opinions  are  a  commonly  used  measure  of  workload  and 
include  psychometr ical ly  defined  rating  scales,  structured  questionnaires,  open- 
ended  questionnaires,  structured  and  unstructured  interviews.  Surprisingly, 
research  on  the  results  of  subjective  measures  indicates  that  these  measures  are 
often  sensitive  and  provide  meaningful  data  to  the  investigator.  This  is 
attributed  to  pilot  acceptance  which  is  generally  favorable  and  also  to  the  fact 
that  opinion  ratings  are  not  intrusive  and  can  be  administered  following  laboratory 
or  field  testing.  No  special  provisions  of  physical  space,  portability,  data 
transmission,  or  integration  into  the  aircraft  system  are  required.  In  most 
cases,  the  subjective  rating  is  used  with  other  measures  of  workload  for  greater 
reliability.  Current  research  in  this  area  is  directed  toward  the  development  and 
evaluation  of  workload-specific  rating  scales. 

SPARE  MENTAL  CAPACITY. 

Another  workload  measurement  technique  which  falls  into  the  general  category  of 
behavioral  measures  of  mental  workload  (Williges  and  Wierwille,  1979)  is  the 
evaluation  of  the  concept  of  spare  mental  capacity.  This  concept  is  based  on  the 
assumption  of  a  limited  channel,  sampling  model  of  the  human  operator  (Rolfe, 
1974).  Spare  mental  capacity  is  the  difference  between  the  total  workload  capacity 
of  the  operator  and  the  capacity  needed  to  perform  the  task: 

Total  workload  -  capacity  needed  to  ■  spare  mental  capacity 
capacity  perform  task 

This  theory  assumes  that  an  upper  bound  exists  on  the  operator's  ability  to 
gather  and  process  information.  Williges  and  Wierwille  (1979)  describe  three 
general  methodological  approaches  for  the  measurement  of  workload  using  the 
spare  mental  capacity  hypothesis.  They  are  (1)  task  analytic,  (2)  secondary 
task,  and  (3)  occlusion  procedures. 

Task  analytic  methods  rely  heavily  on  mathematical/theoret ical  methods  from  the 
field  of  system  engineering,  and  data  are  usually  obtained  through  laboratory  and 
simulation  tests  rather  than  through  actual  flight  evaluation.  The  underlying 
assumption  is  that  all  task  components  performed  serially  require  specific  lengths 
of  time  to  complete.  If  the  actual  time  available  for  completion  exceeds  the  sum 


of  theoretical  time  for  performing  task  components,  the  inference  is  that  spare 
mental  capacity  exists.  Stress  and  task  loading  occur  when  time  13  insufficient  to 
perform  the  tasks.  See  Pew  (1977),  Sheridan  and  Strassen  (1973),  and  Williges  and 
Wierwille  (1979)  for  a  breakdown  of  specific  task  analytic  approaches  and 
me  thodology . 

The  secondary  task  procedures  provide  an  additional  task  for  the  human  operator  to 
perform  when  the  main  (or  primary)  task  has  b^en  attended.  Secondary  task  per¬ 
formance  becomes  an  indirect  measure  of  operator  workload  based  on  the  theory  that 
performance  of  the  additional  task  decreases  as  the  attentional  demand  of  the 
primary  task  increases.  The  various  secondary  task  procedures  have  been  classified 
by  researchers  into  six  general  areas:  (1)  nonadaptive,  (2)  arithmetic  logic, 
(3)  nonadaptive  tracking,  (4)  time  estimation,  (5)  adaptive  arithmetic  logic,  and 
(6)  adaptive  tracking.  See,  fot  example,  Jex  (1977),  Michon  (1966),  Pew  (1977), 
Jex,  McDonnell  and  Phatak  (1966),  Jex  and  Clement  (1977),  O'Donnell  (1975),  Ogen, 
Levine,  and  Eisner  (1979),  and  Gopher  and  North  (1974). 

Some  of  the  criteria  for  selection  of  a  secondary  task  include  the  following 
points:  It  should  not  physically  interfere  with  the  task;  it  should  require  little 
training;  and  it  should  allow  easy  scoring. 

Occlusion  is  similar  to  the  secondary  task  technique  in  that  it  is  a  time-sharing 
technique  and  can  be  used  in  cases  where  primary  informational  inputs  are  visual. 
The  procedure  for  using  occlusion  includes  suppressing  visual  information  inputs. 
For  example,  the  operator  may  wear  a  helmet  or  hat  fitted  with  an  opaque  visor 
which  can  be  closed  by  external  control,  or  the  electronic  displays  can  be  blanked 
ou'  to  accomplish  blocking.  Results  of  driving  tests  where  the  occlusion  method 
was  .^ed  revealed  that  the  less  frequent  the  observations,  the  slower  the  speed  the 
driver  assumed.  The  taster  the  speed,  the  more  numerous  were  the  driver's  obser¬ 
vations,  as  would  be  expected.  Studies  that  used  visual  interruption  to  assess 
driver’s  sensitivity  to  degraded  conditions  found  that  this  method  was  sensitive  to 
task  difficulty  and  operator  skill  (sec  Williges  and  Wierwille,  1978). 

PRIMARY  TASK  MEASURES. 

The  major  underlying  hypothesis  for  the  primary  task  performance  assumes  that  as 
the  mental  workload  of  a  human  operator  increases  the  performance  of  that  operator 
may  change,  usually  in  the  direction  of  degradation.  Such  a  change  is  an  indi¬ 
cation  of  increased  workload.  A  secondary  hypothesis  suggests  that  successful 
completion  of  a  mission  is  a  measure  of  workload  in  itself.  If  a  mission  cannot  be 
completed  successfully,  then  one  can  infer  that  the  operator  is  overloaded. 
Workload  studies  using  primary  task  measures  are  divided  into  three  major 
categories:  single  measures,  multiple  measures,  and  ma tremat  ical  modeling.  For 
studies  utilizing  single  task  measures,  see  Brictson  (1974),  Jex  and  Clement 
(1977),  Senders  (1977),  Welford  (1978),  Wierwille  and  Willigis  (1978).  For  infor¬ 
mation  on  multiple  task  measures,  see  Rolfe,  Chappelow.  Evans,  Landsay  and  Browning 
(1974).  The  greatest  applicability  of  the  primary  task  measures,  whether  single  or 
multiple,  is  in  a  high  workload  situation  as  revealed  by  various  research  findings. 
In  a  low  workload  situation,  primary  task  measures  have  not  been  demonstrated  to  be 
useful  due  to  the  fact  that  the  operator  adapts  to  maintain  output  at  an  acceptable 
level.  Mathematical  modeling  studies  with  workload  implications  are  fairly  recent, 
although  mathematical  modeling  using  dynamic  or  mathematical  equations  of  human 
operator  performance  in  systems  have  a  longer  history  (williges  and  Wierwille, 
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Several  studies  have  examined  describing  motions  and  similar  models  in  manual 
control  systems.  See  Jex ,  McDonnell  ai.d  Phatak  (  1966)  and  Jex  (  1977  )  which 
describe  the  results  of  a  critical  tracking  task  in  conjunction  with  a  describing 
function  model  to  assess  workload. 

PHYSIOLOGICAL  MEASURES. 

In  order  to  measure  workload  using  physiological  parameters,  equipment  used  by 
medical  personnel  has  been  used.  In  addition,  certain  specific  workload  measuring 
devices  have  been  developed.  Wierwille  (1979)  classifies  physiological  measures 
and  names  the  following  single  measures:  flicker  fusion  frequency;  galvanic  skin 
response,  skin  impedance;  electrocardiogram,  phonocardiograro,  pie thysmogram;  heart 
rate,  heart  rate  variability,  and  blood  pressure;  electromyogram;  muscle  tension; 
electroencephalogram;  evoked  cortical  potential;  eye  and  eyelid  movements, 
pupillary  dilation;  respiration  analysis;  body  fluid  analysis;  and  speech  pattern 
analysis.  Multiple  physiological  measures  refers  to  the  combination  of  two  or  more 
techniques.  The  table  below  lists  the  physiological  measures  addressed  in  this 
bibliography  and  the  sources  that  provide  an  in-depth  description  and  analysis. 

TABLE  1.  PHYSIOLOGICAL  TECHNIQUES 


Technique 

Evoked  cortical  potential 
Eye  and  eyelid  movements 

Pupillary  dilation 


Multiple  measures 


Heart  rate,  heart  rate 
variability,  blood  pressure 


Galvanic  skin  response 


Sources 


Spyker,  Stackhouse,  McLane,  Khalifalla  (1971) 
Isreal,  Wickens,  Chesnev  (1980) 

Harris  and  Christhilf  (1980) 

Young  and  Sheena  (19, S) 

Krebs,  Wingert,  Cunningham  (1977) 

Ephrath,  Role,  Stephens,  Young  (1980) 

Hahnemann  (1979) 

Beatty  (1980) 

Krebs,  Wingert,  Cunningham  (1977) 

Poocl;  (  1977) 

McHugh,  Brictson,  Naitoh  (1974) 

Brictson  (1974) 

Boyce  (1974) 

Ettema  and  Zielhuis  (1971) 

Firth  (1973) 

Hicks  and  Soliday  (1977) 

Hahnemann  (1969) 

Mulder  (1977,  1973) 

Rohmert,  Laurig,  Philipp  and  Luczak  (1973) 
Roscoe  (1978) 

Hahnemann  (1974) 

Spyker,  Stackhouse,  Khalafalla  (1971) 
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More  complete  information  on  the  measures  described  above  can  be  obtained  from  the 
sources  mentioned.  In  addition  to  specific  methodology,  many  of  the  sources 
address  limitations  of  practical  application.  The  professional  orientation  of  the 
author  should  be  kept  in  mind,  whether  it  be  physiologist,  psychologist,  operations 
and  research  analyst,  or  systems  engineer.  The  systems  engineer  may  emphasize  a 
particular  aspect  of  workload,  for  example,  the  time  available  to  perform  a  task. 
Psychologi  sts  may  emphasize  the  information  processing  aspects  of  mental  workload 
and  physiologists  may  emphasize  operator  stress  and  arousal  (Williges  and 
Wierwille.)  The  most  inclusive  concept  of  mental  workload  and  its  accurate 
measurement  can  therefore  be  obtained  through  a  combination  of  definitions,  con¬ 
cepts,  and  approaches  from  all  Lhe  disciplines  that  have  investigated  this  topic. 

CONCLUS ION 


It  is  apparent  from  a  review  of  the  current  literature  on  mental  workload  that  no 
single  definition  of  workload  exists.  Rather,  each  investigator  had  to  give  his 
own  definition  which  is  based  on  the  unique  situation  to  be  investigated.  It  is 
also  apparent  that  to  conduct  a  thorough  investigation  a  combination  of  measures  is 
recommended  which  includes  objective,  subjective,  and  physiological  measures. 
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quite  helpful.  Simulation  methods  have  the  potential  of  providing  quite  useful 
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applied,  or  have  potential  application  to  the  problem  of  measurement  of  pilot 
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workload,  especially  as  it  occurs  on  relatively  short  missions.  Selected  studies 
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reviewed  in  relation  to  their  applicability  to  securing  answers  to  operational 
questions.  The  Lypes  of  methods  are  classified  as  laboratory,  analytic  and 
synthetic,  simulator,  and  in-flight.  The  paper  concludes  with  a  general  discussion 
of  relative  merits,  some  of  the  cautions  to  be  observed  in  application  and  in 
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aircraft  cockpit  and  to  ascertain  the  effects  that  the  availability  of  such  infor¬ 
mation  would  have  on  air  traffic  control  procedures  and  performance.  The  program 
was  initiated  in  mid-1970  and  continued  for  a  period  of  5  years.  A  sequence  of 
five  distinct  research  phases  was  carried  out  and  focused  on  various  individual 
applications  of  the  Airborne  Traffic  Situation  Display  (ATSD).  This  report 
presents  an  overview  of  the  entire  effort  and  a  final  assessment  of  the  anticipated 
tole  of  the  ATSD  in  future  air  traffic  control  systems.  Certain  applications  of 
the  device,  such  as  guidance  on  the  airport  surface  and  monitoring  operations  on 
closely-spaced  parallel  runways,  have  not  been  documented  in  previous  reports, 
hence  these  areas  are  discussed  in  greater  detail. 

11.  Eggemeier,  F.T.  Some  Current  Issues  in  Workload  Assessment.  Proceedings  of 
the  Human  Factors  Society  -  24th  Annual  Meeting,  1980.  b69-673. 

The  purpose  of  this  paper  is  to  discuss  (1)  effects  of  operator  strategy  on  the 
levels  of  load  experienced  by  the  operator,  and  (2)  conceptualizations  regarding 
the  nature  of  operator  capacities  and  resources.  Various  current  positions  related 
to  these  Lwo  issues  are  presented  with  implications  for  workload  assessment. 

The  general  conclusions  reached  as  a  result  of  the  literature  search  include  a  view 
of  workload  as  a  multifaceted  or  multidimensional  problem  area  in  which  a  number  of 
factors  combine  and  interact  in  order  to  determine  the  ultimate  workload 
experienced  by  the  operator.  Major  elements  of  workload  included  within  the 
theoretical  framework  include  system  demands,  operator  strategies,  effects  of 
practice  on  the  development  of  internal  anticipatory  models  of  the  system, 
capacities  of  the  operator,  and  effort  expended  by  Lhe  operator  in  performance  of 
Lhe  task.  To  effectively  measure  workload  and  validate  results,  measures  of  each 
of  the  above  elements  need  to  be  developed  including  a  variety  of  performance- 
based,  subjective,  and  physiological  methods  for  assessment  of  workload  which  have 
potential  application  to  major  elements  within  the  framework. 

12.  Ephrath,  A.R.,  Hole,  J.R.,  Stephens,  A.T.,  and  Young,  L.K.,  Instrument 

Scan  —  Is  it  an  Indicator  of  the  Pilot's  Workload?  Proceedings  of  the  Human 
Factors  Society  -  24th  Annual  Meeting,  1980.  237-259. 

Three  NASA  lest  pilots  served  as  subjects  in  these  experiments  which  investigated 
the  relationship  between  an  aircraft  pilot's  visual  scanning  of  instruments  and  nis 


level  of  mental  activity  during  a  simulated  approach  and  landing.  The  subjects 
were  presented  with  a  primary  piloting  task,  an  arithmetic  task  designed  to  control 
mental  loading,  and  a  side  task  for  calibration  of  the  mental  loading  task.  The 
subjects  "look"  point  was  obtained  by  using  a  highly  modified  oculometer  system. 
The  pilot's  eye  scan  of  the  instruments  was  sampled  and  recorded  at  31-millisecond 
intervals. 

Results  of  the  side  task  showed  a  definite  increase  in  workload  when  the  arithmetic 
task  was  introduced.  The  x-y  plots  of  pilot  look  poinL  for  each  Begment  of  the 
approach  also  show  substantial  qualitative  differences  between  the  different  levels 
of  1 oad i ng . 

Preliminary  results  were  obtained  with  fixation-sequences  of  length  k  =  1,2,3 
and  4.  These  suggest  a  monotonic  trend  in  the  value  of  the  entropy  as  the  mental 
loading  increases. 

Further  work  is  being  done  in  which  eye-scanning  data  will  be  calibrated  against 
physiological  measures  of  workload,  and  in  which  the  difficulty  of  the  primary  task 
will  remain  constant  throughout  the  entire  run  rather  than  through  a  segment  (using 
tasks  like  holding  patterns  and  spiral  climbs.) 

13.  Ettema,  J.H.  and  Zielhuis,  R.L.  Physiological  Parameters  of  Mental  Load. 
Ergonomi c  s ,  1971,  14,  137-144. 

In  Lhe  concept  of  mental  load,  it  is  not  the  complexity  of  a  task  but  Lhe  aroounL  of 
information  handled  per  unit  Lime  (the  intensity)  which  determines  the  mental  load, 
creating  a  strong  analogy  between  mental  and  physical  load.  Essential  aspects  of 
the  term  "load,"  as  used  in  work  physiology,  can  be  generalized  to  menial  load  in 
the  following  ways: 

1.  Questions  of  the  physiological  "costs"  (i.e.,  biological  consequences)  of 
a  given  performance. 

2.  External  load  to  be  measured  in  units  of  per formance/ 1 ime  ( signals /mi nut e , 
choices/time) . 

3.  Physiological  changes  correlating  with  the  load. 

4.  Pathological  symptoms  due  to  exLreme  load,  such  as  irritation, 
disorientation,  violent  behavior,  and  psychosomatic  illnesses. 

In  order  to  study  the  effecL  of  several  conditions  of  mental  load  on  hearL  rate; 
i.e.,  s i nu sa r r y t hm i a ,  blood  pressure,  and  brealhing  rate,  an  experiment  was 
conducted  using  a  simple  binary  choice  task  to  induce  mental  load. 

The  recording  of  heart  rate,  s inusarry thmi a  and  breathing  rate  was  continuous;  that 
of  blood  pressure  discontinuous. 

The  results  indicate  that  all  tasks  were  performed  correctly  by  all  of  the  subjects 
when  a  tolerance  of  maximal  3  percent  mistakes  was  accepted.  For  each  of  the 
parameters  measured,  there  appears  to  be  a  rise  in  heart  rale,  systolic  and  dias¬ 
tolic  blood  pressure,  and  breathing  rate,  and  a  suppression  of  the  s inusarrythmi a 
score  during  mental  load.  The  author  states  that  the  findings,  along  with  results 
of  other  experiments  cited,  support  the  hypothesis  that  mental  load  induces  an 
increase  in  the  activity  of  the  sympathetic  nervous  system. 
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14.  Firth,  P.A.,  Psychological  Factors  Influencing  the  Relationship  Between 
Cardiac  Arrhythmia  and  Mental  Load.  Ergonomics ,  1973,  16,  5-16. 

A  survey  of  recent  publications  was  conducted  and  results  suggest  that  the  proposed 
relationship  between  cardiac  arrhythmia  and  mental  load  is  affected  by  the 
following: 

1.  The  absence  of  an  agreed  definition  of  mental  load  makes  interpretation  of 
heart  rate  variability  data  and  its  correlation  with  other  measures  difficult. 

2.  Factors  such  as  dynamic  and  static  workload  respiratory  rate,  emotion  and 
ago  have  boon  shown  lo  influence  heart  rate  variability. 

3.  There  is  more  than  one  method  of  expressing  heart  rate  variability  in 
numerical  terms.  Each  expression  is  formulated  to  emphasize  different  aspects  of 
variability,  such  as  amplitude  or  frequency,  and  also  Lo  differ  as  to  whether  they 
are  weighted  for  mean  heart  rate. 

In  addition,  the  complexity  of  the  psychological  factors  which  may  influence  task 
difficulty  is  discussed  with  reference  lo  second-by-second  changes  in  heart  rate 
variabil ity . 

The  paper  concludes  with  the  following  points:  The  need  for  a  better  understanding 
of  the  task  and  the  relaLed  environmental  and  emotional  factors;  i.e.,  task  content 
and  demand  on  the  individual,  and  the  need  for  a  greater  understanding  of  heart 
rate  variability  as  a  physiological  response. 

13.  Gartner,  W.B.  and  Murphy,  M.R.,  Pilot  Workload  and  Fatigue:  A  Critical  Survey 
of  Concepts  and  Assessment  Techniques.  Moffett  Field,  California:  National 
Aeronautics  and  Space  Administration,  Ames  Research  Center,  NASA  TN  D-83b5,  1976. 

The  report  covers  a  conceptual  analysis  of  pilot  workload  and  fatigue,  an  overview 
of  assessment  techniques,  and  a  critique  of  the  same.  A  discussion  follows  on  the 
management  of  unwanted  workload  and  fatigue  effects  in  the  workload  environment. 
The  authors  constructed  tables  in  which  they  delineate  indicators  of  workload  and 
fatigue.  These  tables  and  references  to  literature,  using  the  particular  indicator 
that  is  ciLed  in  the  tables,  are  included  in  the  appendix  of  this  document. 

The  central  difficulty,  noted  by  the  authors,  was  the  lack  of  any  widely  accepted 
and  applied  criteria  for  establishing  functional  relationships  between  indicators 
and  concepts.  That  is,  measures  of  fatigue  and/or  workload  are  selected  on  the 
basis  of  theoretical  interest,  as  opposed  to  relevancy  in  a  particular 
conceptualization. 

Despite  conceptual  and  practical  difficulties,  however,  attempts  to  develop  and 
apply  useful  measures  of  pilot  workload  are  being  vigorously  pursued.  Several 
alternative  concepts  and  approaches  to  the  problems  are  reviewed,  with 
recommendations  for  followup. 
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16.  Gawron,  V.J.,  The  Effects  of  Noise:  A  Review.  Proceedings  of  the  Human 
Factors  Society  ~  24th  Annual  Meeting  -  1980,  233-237. 


The  el  feel  ot  noise  on  performance  is  generally  contradictory.  Some  studies 
indicate  that  noise  degrades  performance,  others  show  no  effect,  while  others 
demonstrate  a  facilitation  of  performance.  Relevant  research  in  the  field  of 
noise  and  its  effect  is  presented  and  reviewed  in  this  paper.  Studies  reviewed 
suggest  that  noise  effects  are  complex  and  may  be  influenced  by  many  variables. 
Subject,  task,  measurement,  and  noise  characteristics  may  all  interact  to  produce 
the  notable  inconsistency  in  results.  The  author  summarized  her  findings  of  the 
literature  review  to  say  that  the  final  explanation  may  be  a  compromise  between 
previously  opposing  points  of  view. 


17.  Gerathewohl,  S.J.,  Brown,  E.L.,  Burke,  J.E.,  Kemball,  K..A. ,  Lowe,  W.F. ,  and 
Stackhouse,  S.P.,  In-flight  Measurement  of  Pilot  Workload:  A  Panel  Discussion. 
Aviation,  Space,  and  Environmental  Medicine,  June  1978,  810-822. 

A  group  of  United  States  scientists  engaged  in  in-flight  measurements  of  pilot 
workload  discussed  the  problems  and  aspects  of  workload  (including  the  techniques 
used  for  in-flight  measurements)  and  various  workload  models  (including  design- 
oriented,  operational,  psychological  and  physiological  concepts).  Different 
experimental  approaches  were  discussed.  Experiences,  results,  and  further  plans  as 
identified  during  the  48th  Annual  Scientific  Meeting  of  the  Aerospace  Medical 
Association  in  Las  Vegas,  Nevada,  May  10,  1977,  were  also  discussed.  Contributions 
by  each  presenter  are  summarized. 

18.  Gerathewohl,  Ph.D.  J.,  Definition  and  Measurement  of  Perceptual  and  Mental 
Workload  in  Aircrews  and  Operators  of  Air  Force  Weapons  Systems:  Status  Report. 
Proceedings  of  the  AGARD  Conference  on  Higher  Mental  Functioning  in  Operational 
Environments.  AGARD-181 ,  October  1975,  Cl-l-Cl-7. 

Dr.  Gerathewohl  accepts  the  definition  of  pilot  workload  given  by  Jahns  (1972)  who 
defines  workload,  in  his  review  of  the  subject,  as  "...an  integrative  concepL  for 
evaluation  of  the  effects  on  the  human  operator  associated  with  the  multiple 
stresses  occurring  within  man-machine  environments."  Further,  he  divided  the 
concept  into  three  functionally-related  components:  (1)  input  load,  (2)  operator 
effort,  and  (3)  work  results. 

Various  methods  of  measuring  pilot  and  aircrew  workload  using  psychological, 
physiological,  and  operational  criteria  are  covered,  including  a  summary  of  some 
results  obtained  from  studies  utilizing  various  techniques.  In  spice  of  some 
valuable  results,  workload  measurements  associated  with  highly  complex  and 
demanding  conditions,  including  the  introduction  of  new  and  sophisticated 
equipment,  remains  a  difficult  task. 

19.  Glines,  C.V.,  A  Statesman-Like  Challenge.  Air  I ine  Pilot ,  1978,  47,  6-13. 

This  article  is  a  report  on  an  ALPA-sponsored  symposium  entitled  "Man-Machine 
Interface:  Advances  in  Workload  Study"  which  was  the  first  initiative  within  the 
aviation  community  to  bring  psychologists,  physiologists,  and  other  biomedical  and 
behavioral  specialists  to  explore  the  basics  of  workload  and  possible  applications 
to  the  cockpit  environment.  One  of  the  issues  addressed  at  the  conference  was  a 
definition  of  workload  and  acceptable  ways  of  measuring  it. 
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20.  Goerres,  H.  ,  Subjective  Stress  Assessment  as  a  Criterion  for  Measuring  the 
Psychophysical  Workload  on  Pilots.  Proceedings  of  the  AGARD  Conference  on  Methods 
to  Assess  Workload.  ACARD-CP-2 16,  1977,  Bll-l-Bll-8. 

The  author  first  defines  the  term  "psychophysical  workload"  as  comprising  the 
effects  of  the  "total"  workload  on  the  human  organism,  human  behavior,  and 
subjective  feeling.  Psycliophy sical  workload,  induced  by  an  activity,  will  not  only 
depend  upon  duration  and  intensity  of  stressing  stimuli,  but  also  upon  intra¬ 
individual  factors  in  the  stressed  subject  himself  (physical  features,  functioning 
of  the  sensory  organs,  vegetative  status  and  present  state  of  health  as  a  pre¬ 
requisite  to  physical  performance;  job  related  knowledge,  abilities,  skills,  need 
for  achievement,  experience,  and  emotional  stress  resistance  as  psychic  and  mental 
determinants  of  strain). 

Time  constraints  allowed  only  the  use  of  subjective  assessments  of  workload 
situations  through  standardized  interviews  and  questionnaires  administered  to  217 
pilots;  including  jet  pilots,  multiple  engine  prop  pilots,  single  engine  prop 
pilots,  and  helicopter  pilots. 

The  author  readily  points  out  that  subjective  opinions  cannot  be  a  substitute  for 
objective  scientific  observations  of  measurements,  but  dc  provide  a  useful  source 
of  information,  especially  as  an  adjunct  to,  in  cases  where  scientific  measuring  is 
impossible  for  any  reason. 

Findings  indicate  that  the  physical  workload  in  its  entirety  is  unanimously 
assessed  lower  by  pilots  than  mental  and  emotional  "psychological"  workload. 

21.  Gopher,  D.  ,  and  North,  R.A.  ,  The  Measurement  of  Attention  Capacity  Through 
Concurrent  Task  Performance  with  Individual  Difficulty  Levels  and  Shifting 
Priorities.  Illinois  University,  Aviation  Research  Lab,  ARL-74-13,  1974. 

Unsolved  problems  in  the  application  of  secondary  task  techniques  include  the 
evaluation  of  relative  changes  in  performance  in  dual  task  situations,  the 
prediction  of  possible  interactions  between  different  tasks  and  their  components, 
and  the  extent  of  voluntary  control  of  capability  allocation.  This  article 
describes  a  three-part  experiment  in  which  an  effort  was  made  to  address  these 
problems  by  a  new  methodological  approach.  The  three  successive  phases  included 
(1)  separate  performance  of  the  experimental  casks  with  adaptive  adjustment  of 
difficulty,  (2)  simultaneous  performance  of  the  tasks  wiLh  equal  task  priorities, 
and  (3)  simultaneous  performance  with  several  manipulations  of  the  two  task 
priorities.  The  tasks  used  for  the  experiment  were  a  single-axis,  compensatory 
tracking  task  and  a  digit-processing,  reaction  time  task. 

22.  Gunning,  D.,  The  Measurement  of  Aircrew  Task-Loading  During  Operational 

Flights.  P  roceedings  of  the  Human  Factors  Society  24th  Annual  Meeting,  1980, 

249-2  32.  . .  ~  “ 

The  use  of  task  analytic  techniques,  as  methods  for  assessing  operator  workload,  is 
discussed.  Task-loading  data  were  collected  during  four  operational  flights  of  the 
Air  Force's  K.C-135  tanker  aircraft,  using  a  Datamyte  data  recorder  which  observers 
used  to  record  the  occurrences  of  45  different  tasks,  and  performance  by  the  pilot, 
copilot,  and  navigator.  (The  Datamyte  is  a  hand-held  unit  with  keyboard  and  32K  of 
solid-state  memory,  ami  has  several  input  modes.) 
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i  The  raw  data  were  summarized  by  computing  the  percentage  of  Lime  spent  on  each 

)  task.  Task-loading  profiles  for  each  crew  member  and  peak  workload  situations  that 

occurred  are  presented.  The  author  concludes  that  the  task-loading  measurement 
j  techniques  produced  useful  data  especially  with  reference  to  the  causes  of  high 

task  loading.  Likewise,  the  limitations  of  the  procedure  are  cited,  with  the 
^  subjective  nature  of  the  observer  recording  the  data  being  the  most  serious. 

23.  Harris,  R.L.,  and  Christhilf,  D.M.,  What  Do  Pilots  See  in  Displays? 
Proceedings  of  the  Human  Factors  Society  -  24th  Annual  Meeting,  1980,  22-27. 

Based  on  pilots  scanning  data  and  discussions  with  pilots,  general  aviation  flight 
instruments  have  been  classified  into  three  categories.  The  classification  is 
related  to  the  type  of  information  presented,  the  way  the  information  is  used,  and 
the  pilot's  role  as  a  monitor  and  controller.  Suggestions  are  made  for  modifying 
the  instruments  of  one  category  to  improve  the  information  displayed,  so  the  pilot 
i  can  more  quickly  extract  the  information  needed. 

j 

24.  Hart,  S.G.,  and  Wempe,  T.E.,  Cockpit  Displays  of  Traffic  Information:  Airline 
Pilots  Opinions  About  Content.  ,  Symbology,  and  FormaL.  National  Aeronautics  and 

i  Space  Administration,  Ames  Research  Center,  NASA-TM-7860 1  ,  1979. 

A  number  of  candidate  computer-generated  cockpit  displays  of  traffic  information 
displays  and  displays  options  were  simulated  statically  and  were  shown  to  23 
I  airline  pilots  who  were  asked  to  respond  to  more  than  250  questions  about  them. 

The  pilots  indicated  that  the  amount  and  complexity  of  navigation  information 
displayed  should  increase  with  altitude  and  map  scale.  Terrain  information  should 
i  appear  automatically  if  a  pilot's  own  aircraft  descenas  below  the  minimum  9afe 

■  altitude  and  should  include  only  those  obstructions  within  2,000  feet  or  less.  Few 

i  pilots  felt  that  weather  information  should  be  displayed  on  a  CDT1,  but  if  it  was, 

i  it  should  be  a  pilot  request  only.  A  Chevron-shaped  symbol,  located  so  that  the 

majority  of  the  map  area  was  ahead,  was  preferred.  The  position,  altitude,  ground 
,  speed,  ground  track,  weight  class,  and  flight  path  history  of  other  aircraft  should 

be  presented  graphically  by  coding  the  shape  of  the  symbol  for  other  aircraft  or 
|  presented  digitally  in  data  tags  displayed  at  pilot  request.  All  pilots  thought 

that  color  coding  was  necessary  to  recognize  different  categories  of  information 
quickly  and  accurately.  The  majority  of  pilots  felt  thaL  a  CDT1  would  provide 
'  useful  information  even  though  its  presence  might  increase  their  workload, 

!  particularly  during  its  introductory  stages. 

I 

25.  Hart,  S.G.,  and  Loomis,  L.L.,  Evaluation  of  the  Potential  Format  and  Content 
of  a  Cockpit  Display  of  Traffic  Information.  Human  Factors  1980,  22  (5)  591-604. 


Airline  pilots  and  instrument  rated  general  aviation  pilots  served  as  test 
subjects.  The  initial  approach  was  to  solicit  their  opinions  about  display  format, 
information  content,  and  symbology  after  viewing  more  than  100  candidate  displays. 

Results  indicate  that  it  is  more  difficult  and  apparently  requires  more  time  to 
„  evaluate  the  vertical  relationship  between  two  targets  than  the  horizontal 

relationship.  Display  features  that  might  contribute  to  a  pilot's  ability  to 
f  perceive  the  traffic  situation  correctly,  such  as  flight  path  predictors,  were 

'  j  identified.  Other  display  features  for  which  pilots  expressed  a  preference  in  the 

!fe- 

i 


1  5 


first  experiment  (such  as  ground  speed  and  climb/descend ,  arrows  presented  in  a 
data  tag,  and  relative  altitude  encoding  of  symbols  for  other  aircraft)  did  not 
contribute  to  improved  performance,  speed,  or  accuracy. 

The  authors  concluded  that  additional  research  is  required  to  determine  the  best 
way  to  inform  a  pilot  about  the  vertical  relationsip  between  his/her  own  and 
another  aircraft.  Solutions  for  further  study  are  suggested. 

26.  Hicks,  J.A.,  and  Soliday,  S.M.,  An  Evaluation  of  Sinusarrythmia  as  a  Measure 
of  Mental  Load.  Proceedings  of  the  Human  Factors  Society  21st  Annual  Meeting, 
October  17-20,  1977,  191-196. 

The  main  purpose  of  this  paper  is  to  provide  an  evaluation  and  validation  of 
sinusarrythmia  as  a  measure  of  mental  workload.  Sinusarrythmia  refers  to  the 
irregularity  in  the  length  of  the  interbeat  intervals  found  in  the  heart  rate 
pattern  of  a  normal  individual  sitting  at  rest.  Research  in  the  past  indicated 
that  this  irregularity  disappears  as  a  function  of  the  mental  load  imposed  on  the 
individual.  A  secondary  purpose  of  the  paper  was  to  compare  the  sinusarrythmia 
approach  to  estimating  mental  workload  with  a  secondary  task  loading  technique.  A 
time-interval  production  task,  presented  by  Michon  (1966)  served  as  the  secondary 
task. 

Results  validated  basic  findings  reported  by  Kalsbeek  (1971)  that  sinusarrythmia 
tends  to  be  suppressed  as  a  function  of  mental  workload.  An  important  aspect  of 
this  is  that  once  the  mental  workload  task  reaches  such  a  level  that  it  causes  a 
complete  suppression  of  sinusarrythmia  it  reappears,  and  the  individual  will  begin 
to  make  errors  unless  there  is  a  reduction  in  the  mental  load  within  a  few 
minutes . 

27.  Isreal,  J.B.,  Wickens,  D.,  and  Chesney,  G.L.,  The  Event-Related  Brain 
Potential  (ERP)  as  an  Index  of  Display-Monitoring  Workload.  Human  Factors, 
1980,  22,  211-224. 

The  ERP  is  a  transient  series  of  voltage  oscillations  in  the  brain  that  can  be 
recorded  on  the  scalp  in  response  to  any  discrete  stimulus  event.  The  ERP  waveform 
can  be  analyzed  into  a  series  of  components  which  index  the  sequence  of 
information-processing  activities  triggered  by  the  eliciting  stimulus.  The  magni¬ 
tudes  and  latencies  of  the  components  have  been  shown  to  vary  as  a  function  of  the 
physical  and  informational  properties  of  the  eliciting  event,  as  well  as  the 
subject's  cognitive  response  to  the  stimulus. 

This  article  describes  the  use  of  the  ERP  as  an  index  of  the  task  workload  through 
the  discussion  of  an  experiment  in  which  perceptual  load,  incurred  by  monitoring  a 
simulated  air  traffic  control  display  for  discrete  events,  is  assessed.  The  ERP 
measures  were  found  to  systematically  reflect  differences  in  task  workload  and  to 
covary  closely  with  the  reaction  time  measure. 

28.  Jacquot,  B.,  Electronic  Displays  Can  Reduce  Cockpit  Overload.  ICAO  Bulletin, 
1979,  12-27. 

The  article  describes  tests  by  the  French  government  of  heads-up  and  heads-down 
displays,  showing  both  advantages  and  drawbacks  of  such  advanced  cockpit 
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pt1  ese m at i ons .  Among  the  advantages  of  incorporating  electronic  displays  in  the 
cockpit,  the  author  describes  a  reduction  of  workload  as  a  major  one.  Current 
cockpit  conf igurations  involve  a  proliferation  of  instruments.  This  leads  to  a 
dispersion  of  piloting  parameters  which  requires  t ime- consuming  visual  scanning  by 
the  pilot.  In  addition,  data  provided  by  conventional  instruments  are  not  always 
suitable  for  flying  in  normal  conditions  and  are  even  less  suitable  for  solving 
special  problems  that  may  arise.  The  conventional  instruments  fail  to  give  the 
pilot  an  overall  view  which  enables  him  to  recognise  and  respond  to  the  situation. 


29.  Jex ,  H.R.,  A  Proposed  Set  of  Standarized  Subcritical  Tasks  for  Tracking 
Workload  Calibration  in  Moray,  N.  (Ed.)  Mental  Workload:  Its  Theory  and 
Measurement.  New  York  Plenum  Press,  1977,  179-188. 


A  set  of  "subcritical"  tracking  tasks  are  proposed  as  one  portion  of  a  battery  of 
different  types  of  tasks,  each  with  graded  levels  of  mental  workload,  against  which 
investigators  could  calibrate  and  validate  their  measures  of  workload.  The 
recommended  task  description  is  provided,  along  with  mechanization  (including 
controlled  element,  display,  and  controls),  optional  inputs,  operating  procedures, 
and  evaluation  criteria. 


The  author  cuutions  that,  despite  many  years  of  research  on  critical  task 
development  and  application,  the  paper's  recommendations  must  be  considered 
tentative. 

30.  Jex,  H.R.,  McDonnell,  J.D.,  and  Phatak,  A.V.,  A  Critical  Tracking  Task  for 
Manual  Control  Research.  IEEE  Transactions  on  Human  Factors  in  Electronics,  Volume 
HFE-7,  1966,  138-145. 

"A  'critical'  tracking  task  is  developed  in  which  a  human  operator  is  required  to 
stabilize  an  increasingly  unstable  first  order  controlled  element  up  to  the 
critical  point  of  loss  of  control.  Servo  theory  and  operator-describing  function 
measurements  are  used  to  validate  the  basic  assumptions,  and  an  automatically  paced 
critical  task  is  described.  The  results  show  that  the  task  does  constrain  the 
operator's  behavior  as  intended  and  the  critical  instability  depends  primarily  on 
the  operator's  effective  Lime  delay  while  tracking.  A  number  of  applications  for 
the  critical  task  are  reviewed,  including  secondary  workload  research,  control,  and 
measurement  of  operator  and  controlled  element  gain,  and  display  research." 

31.  Jex,  H.R.  and  Clement,  W.F.,  Defining  and  Measuring  Perceptual  Motor  Workload 
in  Manual  Control  Tasks.  In  N.  Moray  (Ed.)  Mental  Workload:  Its  Theory  and 
Measurement .  New  York:  Plenum  Press,  1977,  125-177. 

Despite  the  lack  of  theory  available  to  offer  relevant  answers  to  the  definition 
and  measurement  of  perceptual  motor  workload  ( PML ) ,  a  significant  body  of 
literature  exists  on  pilot  workload.  The  underlying  theme  in  this  literature 
suggests  that  "the  more  unexpected  items  that  must  be  handled  per  unit  time,  the 
higher  the  'mental  workload,'  up  to  a  point  where  no  further  workload  can  be 
handled."  The  authors  interpret  these  observations  about  pilot  workload  to  make 
the  following  hypotheses: 


17 


1.  Operational  manual  control  systems  are  typically  designed  and  developed  to 
require  far  less  chan  the  operator  or  binary  limiting  capabilities;  therefore, 
performance  degradation  due  to  environmental  stress  or  workload  is  seldom  observed, 
except  in  emergencies. 

2.  Intrinsic  ability  (skill,  workload)  limits  can  be  measured  only  under  high 
stress  conditions;  but  the  operator's  capabilities  improve  with  habituation  to  such 
conditions . 

3.  Workload  or  skill  decrements  due  to  exogenous  stresses  are  most  apparent 
when  the  operator  is  near  his  limiting  performance. 

A.  Specialized  control  Lasks  or  surrogates  for  control  tasks  can  be  designed 
to  emphasize  particular  skill  or  workload  factors. 

This  paper  offers  support  for  these  hypotheses.  First,  various  terms,  such  as 
stress,  strain,  and  PML  workload  are  defined  and  illustrated.  Following  a  dis¬ 
cussion  of  the  interacting  conflicts,  an  operator  must  tesolve  in  a  high  workload 
manual  control  situation,  the  concept  of  a  "netacontrol  system"  is  reintroduced;  a 
system  which  governs  operation  of  the  human's  multiprocessor,  time-shared  routines. 
A  series  of  subsections  on  controlling  and  measuring  PML  follows  with  numerous 
examples  given.  Summaries  of  some  are  given  in  tabular  form.  Some  of  the  author's 
relevant  work  (cross-coupled  instability  task  and  psychophy siological  measures)  is 
discussed  in  greater  deail.  Suggested  needs  for  further  research  concludes  the 
study . 

32.  .Johannaen,  G.,  Workload  and  Workload  Measurement.  In  Moray,  N.  (Ed.)  Mental 
Workload:  Its  Theory  and  Measurement.  New  York:  Plenum  Press,  1977,  3-11. 

In  an  effort  to  define  workload,  the  author  divides  operator  workload  into  three 
functionally  relatable  attributes;  input  load,  operator  effort,  and  performance  at 
work  result.  (See  figure  1.) 

The  measurement  of  workload  is  then  described.  In  order  to  assess  the  relative 
value  of  different  workload  measures,  one  has  to  distinguish  between  different 
loading  factors  and  between  different  types  of  effort  expenditure  (scanning  effort, 
perception  effort,  decision  effort,  memory  recall  effort,  etc.). 

An  approach  which  permits  integration  of  most  of  these  effort  measures  is  given  by 
the  time-line  analysis  which  calculates  percent  workloading  for  different  channels 
of  the  human  operator.  In  addition  to  a  time-line  type  integrative  workload 
analysis,  operator  activation-level  measures  should  be  taken  to  assess  peak  load 
situations.  Subjective  effort  ratings  should  be  used  in  any  case,  as  it  is  often 
the  most  reliable  overall  measure. 
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FIGURE  1 .  ATTRIBUTES  OF  OPERATOR  WORKLOAD  AS  DELINEATED  BY  JOHANNSEN 


33.  Kahneraann,  D.,  Pupillary,  Heart  Rate,  and  Skin  Resistance  Changes  During  a 
Mental  Task.  Journal  of  Experimental  Psychology,  1969,  Volume  79,  No.  I,  164-167. 

This  paper  describes  an  experiment  in  which  the  subjects  performed  a  paced  mental 
task  at  three  levels  of  difficulty  while  recordings  of  pupil  diameter,  heart  rate, 
and  skin  resistance  were  made.  Precise  tracking  of  the  changing  effort  during 
performance  of  a  mental  task  was  not  a  unique  characteristic  of  the  pupil  in  this 
study,  although  the  pupillcry  response  yielded  the  most  consistent  results. 

The  fidelity  with  which  the  autonomic  variables  monitored  cask  demands  in  this 
study  can  most  likely  be  attributed  to  the  use  of  a  paced  task  and  timelocked 
response  measurement.  This  approach  contrasts  with  the  more  traditional  psycho- 
physiological  method  of  sampling  autonomic  variables  over  extended  time  periods  and 
examining  mean  differences  between  experimental  conditions  in  overall  levels  of 
activity.  It  is  concluded  that  there  is  independent  behavioral  evidence  that  the 
pattern  of  autonomic  activity  observed  here  corresponds  to  the  load  imposed  by  the 
task . 

34.  Kalsbeek,  J.W.,  Objective  Measurement  of  Mental  Workload:  Possible 
Applications  to  the  Flight  Task.  Proceedings  of  Che  AGARD  Conference  on  Problems 
of  the  Cockpit  Environment,  AGARD-CP-55 .  1970,  4-1-4-3 . 

The  problem  of  objective  measurement  of  mental  workload  is  studied  by  physiological 
and  pyscho logical  methods.  This  paper  addresses  some  methods  and  techniques, 
together  with  their  application  in  simulation  experiments  in  order  to  evaluate 
cockpit  workload.  The  measures  used  include  heart  rhythm,  regularity  of  the  heart 
rate  pattern,  and  distraction  stress. 
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35.  Krebs,  M.  ,  Wingert,  J.,  and  Cunningham,  T.  ,  Exploration  of  an  Ocul oaeter-Based 
Model  of  Pilot  Workload.  National  Aeronautics  and  Space  Administration, 
Contractor's  Report  NASA-CR-145 1 53 ,  1977. 

Potential  relationships  between  eye  behavior  and  pilot  workload  are  discussed.  A 
Honeywell  Mark  LLA  oculometer  was  used  to  obtain  the  eye  data  in  a  fixed-base 
transport  aircraft  simulation  facility.  The  data  were  analyzed  to  determine  those 
parameters  of  eye  behavior  which  were  related  to  changes  in  level  of  task 
difficulty  of  the  simulated  manual  approach  and  landing  on  instruments.  A  number 
of  trends  and  relationships  between  eye  variables  and  pilot  rating  were  found.  A 
preliminary  equation  was  written  based  on  the  results  of  a  step-wise  linear 
regression.  High  variability  in  time  spent  on  various  instruments  was  related  to 
difference-,  in  scanning  strategy  among  pilots.  A  more  detailed  analysis  of 
individual  runs  by  individual  pilots  was  performed  to  investigate  the  source  of 
this  variability  more  closely.  Results  indicated  a  high  degree  of  intrapilot 
variability  in  instrument  scanning.  No  consistent  workload-related  trends  were 
found.  Pupil  diameter  which  had  demonstrated  a  strong  relationship  to  task 
difficulty  was  extensively  reexamined. 

36.  Kreifeldi,  J.G.,  Cockpit  Displayed  Traffic  Information  and  Distributed 
Management  in  Air  Traffic  Control .  Human  Factors ,  1980,  22(6),  P71-691. 

The  feasibility  of  graphically  displaying  traffic  information  in  the  cockpit  on  a 
CRT  is  suggested  as  a  viable  method  for  improving  the  safety  and  orderliness  of  the 
air  traffic  control  system  by  distributing  some  of  its  management  to  pilots 
equipped  with  CDT1,  Three  experiments  conducted  at  NASA-Ames,  using  the  three-cab 
simulator  facility,  are  described.  The  experiments  evaluated: 

1.  Modes  of  distributed  management. 

2.  Management  of  mixed  CDTI/non-CDT 1  traffic- 

3.  Distributed  management  and  curved  approaches. 

The  primary  aim  was  to  assess  manual,  verbal,  visual,  and  Lotal  workloads,  using 
both  objective  and  subjective  measures.  The  results  showed  that  pilots  on  the 
average  preferred  the  distributed  mode  of  management,  contrary  to  the  controllers. 

Distributed  management  based  on  CDT1  produced  no  detrimental  behaviors  and 
exhibited  several  highly  desirable  cliarac ter i s L ic s ,  such  as  large  reductions  in 
controller  verbal  workload  and  smaller  interarrival  time  dispersions  than  that 
achieved  in  centralized  management. 

The  results  of  the  third  experiment  were  the  same  except  that  controllers  also 
expressed  preference  for  distributed  management,  perhaps  reflecting  the  extreme 
difficulty  of  the  vectoring  task  in  this  exercise.  Pilots  will  generally  report  a 
large  visual  workload  with  CDT1,  but  still  prefer  distributed  management, 
indicating  that  any  additional  workload  caused  by  CDT1,  is  more  than  offset  by  the 
additional  information  and  capability  it  provides. 

Controller  workload  reductions  and  reduced  dispersion  tiroes  that  are  common  with 
CDTl-based  distributed  management  indicate  increased  controller  productivity  and 
runway  capacity.  CDTI  also  exhibits  the  potential  ability  to  cope  witn  unplanned 
cont  igenc  i  s  in  which  the  speed  and  quality  of  response  are  essential.  The  "Who's 
in  charge"  syndrome  was  not  observed  in  the  three  experiments. 


20 


37.  McHugh,  LCDR  W.B.,  Brictson,  C.A.,  and  Naitoh,  P.,  Emotional  and  Biochemical 
Effects  of  High  Workload.  Proceedings  of  the  AGARD  Conference  on  Simulation  and 
Study  of  High  Workload  Operations.  AGARD-CP- 146 ,  1974.  A12-2-A12-9. 

"...A  preliminary  longitudinal  mu  1 1 i f ac to ri al  study  of  the  interrelationships  of 
hiocht'ini  cal  ,  mood ,  biographical  factors  and  landing  performance  under  high  workload 
conditions  was  carried  out  with  U.  S.  Naval  Aviators.  Levels  of  serum  cholesterol, 
serum  uric  acid,  blood  lactate,  pyruvate,  and  mood  assessments  were  made  during 
periods  of  nonflying  activity  and  during  periods  of  increased  cumulative  workload. 
Uric  acid  values  fell  during  moderate  cumulative  workload,  and  cholesterol  values 
fell  during  high  cumulative  workload.  Increased  variability  of  pyruvate  and 
lactate  were  noted  with  increased  cumulative  workload.  Increased  workload  did  not 
affect  emotions  or  performance  but  altered  mood  association  patterns  and  alcered 
the  relationships  of  mood  and  performance.  Experience  was  correlated  with  per¬ 
formance  under  zero  cumulative  workload  conditions.  Emotion  correlated  with  high 
performance  under  high  cumulative  workload  conditions." 

38.  Michon,  J.A.,  Tapping  Regularity  as  a  Measure  of  Perceptual  Motor  Load. 
Ergonomics ,  1966,  9,  401-412. 

Secondary  task  may  interfere  functionally  with  the  main  task,  without  necessarily 
making  use  of  the  same  peripheral  pathways.  The  experiments  described  in  this 
article  take  the  above  point  into  account,  and  describes  test  situations  in  which 
the  subject  performs  his  normal  task,  and  simultaneously  taps  a  pedal  as  regularly 
as  possible,  at  a  constant  race  of  about  one  or  two  taps  per  second. 

The  measure  of  Lask  performance  is  Che  average  irregularity,  and  the  basic 
hypothesis  is  that  tapping  performance  will  become  less  regular  when  the  main  task 
perceptual  motor  load  increases.  Though  it  is  a  very  simple  task  to  perform,  it  is 
very  sensitive  to  disruptions  in  timing. 

39.  Moray,  N. ,  Models  and  Measures  of  Mental  Workload.  In  Moray,  N.  (Ed.)  Mental 
Workload:  Xcs  Theory  and  Measurement.  New  York:  Plenum  Press,  1977,  13-21. 

The  extent  to  which  a  man  is  loaded  is  a  function  not  merely  of  the  man,  but  of  the 
task-specific  situation  in  which  he  finds  himself.  There  are  as  many  measures  as 
ttiere  are  types  of  task,  and  the  hope  for  a  unified  theory  of  load  is  a  false  one. 

This  paper  discusses  several  measures,  categorized  as  normative,  empirical  and 
physiologic.',  each  capable  of  evaluating  distinct  parameters,  as  well  as  over¬ 
lapping  ones.  The  need  exists,  however,  to  know  the  relation  between  them,  the 
range  over  which  they  can  be  used,  ("boundary  conditions"),  and  the  population 
statistics  on  individual  differences. 

Problems  of  methodology  and  measurement  as  well  as  recommendations  for  future  study 
regarding  the  formal  relation  between  normative  measures,  and  a  corresponding 
empirical  analysis  of  the  in r err el  at  i  on  of  the  scores  of  the  different  empirical 
measures  (including  the  physiological  ones)  are  discussed. 
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40.  Mulder,  G.,  Sinusarrylhmia  and  Mental  Workload.  In  N.  Moray,  (Ed.)  Mental 
Workload:  Its  Theory  and  Measurement.  New  York:  Plenum  Press,  1977,  327-343. 


Si nusarrythmi a  is  a  variability  in  heart  rate,  and  this  variability  has  been 
associated  with  respiration.  Recently,  more  insight  into  the  mechanisms  underlying 
this  phenomenon  has  been  obtained,  and  this  paper  summarizes  these  mechanisms. 
Spectral  analysis  of  s  i  nusar r hy thmi a  has  been  used,  revealing  the  different 
mechanisms  involved.  Different  techniques  have  been  used  in  order  to  obtain  an 
equidistant  time  series.  This  time  series  is  subsequently  Fourier  transformed 
either  by  the  indirect  method  or  the  direct  one. 

41.  Murrell,  J.F.,  Pilots  Assessment  of  Their  Cockpit  Environment.  Proceedings  of 
the  AGAHD  Conference  on  Problems  of  the  Cockpit  Environment.  AGARD-CP-55,  1970. 
17-1-17-6. 

This  study  utilized  a  postal  survey  method  in  developing  a  technique  whereby 
opinions  of  many  pilots  could  be  collated  and  used.  The  author  notes  a  low 
response  rate;  i.e.,  8.5  percent.  Therefore,  this  study  of  the  responses  of 

airline  pilots  to  open-ended  questions  is  regarded  as  a  preliminary  i nvest igat ion . 
Two  hundred  and  twenty-nine  civilian  airline  pilots  responded  to  four  open-ended 
questions  (concerned  with  design  and  positioning  of  instruments,  displays,  and 
controls  of  the  aircraft  they  flew)  and  one  general  question.  The  survey  covered 
nine  different  aircraft  types.  Results  were  subjected  to  a  content  analysis  and 
classified  according  to  the  item  mentioned,  Nonparametr ic  tests  were  used  (.1)  to 
find  the  degree  of  agreement  between  the  pilots  as  to  the  relative  importance  of 
the  different  questions  and  (2)  to  dispose  of  the  suggestion  that  the  different 
frequencies  of  response  and,  thus,  agreement  was  generated  by  the  fixed  question 
order.  Results  indicate  that  the  pilol'9  main  concerns  are  with  the  design 
adequacy  of  his  communications  and  navigation  equipment  and  the  instruments 
presenting  flight  and  engine  information  and  with  his  own  comfort  and  freedom  of 
movement.  The  author  cautions  against  generalizations  from  the  results  because  of 
the  low  response  rates  involved.  However,  f he  study  does  demonstrate  that  civil 
airline  pilots  are  particularly  concerned  about  certain  areas  of  their  cockpit 
environment,  and  they  agree  about  these  concerns  despite  the  fact  thaL  they  fly 
different  types  of  aircraft. 

42.  Mulder,  G.,  Mental  Load  and  the  Measurement  of  Heart  Rate  Variability. 
Ergonomi e s ,  1973,  16,  69-83. 

Results  of  previous  experiments  have  noted  that  heart  rate  variability  (HV) 
diminishes  during  performance  on  a  binary  choice  task.  These  experiments  were 
repeated  in  the  current  study  in  order  to  find  the  most  sensitive  measure  chat 
would: 

1.  Discriminate  between  the  task  levels  (20,  30,  40,  50,  anu  60  binary 
choices  per  minute). 

2.  Be  stable  over  days  and  rest  periods  (within  ihe  experiment). 

3.  Show  the  least  intersubject  variance. 

Analysis  of  variance,  carried  out  over  raw  scores  in  the  task  conditions,  showed 
all  measures  were  significantly  influenced  bv  the  task  conditions.  The  load  in  this 
experiment  was  the  number  of  binary  choices  per  minute. 
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Spectral  analysis  of  heart  rate  variability  revealed  the  existence  of  a  frequency 
component  at  about  0.10  hertz  (Hz),  a  respiration  frequency,  and  sometimes  a  task 
frequency.  In  a  number  of  tasks,  the  respiration  frequency  increased,  and  the 
amplitude  of  energy  in  the  lower  frequencies  decreased  (decrease  in  HV ) . 

In  future  research,  more  attention  should  be  paid  to  the  type  of  task,  and  the 
behavioral  mechanisms  involved,  and  to  their  presentation  mode.  Also,  it  is 
recommended  that  in  order  to  arrive  aL  a  valid  interpretation,  measurement  of 
relative  thorax  volume  and  blood  pressure  are  indicated.  Analysis  of  single 
subjects  is  recommended  due  to  large  differences  between  subjacts. 

43.  O'Conner,  S. ,  Palmer,  E. ,  BaLy ,  D.,  and  Jago,  S.,  The  Effecl  of  Viewing  Time, 
Time  to  Encounter,  and  Practice  on  Perception  of  Aircraft  Separation  on  a  Cockpit 
Display  of  Traffic  Information.  National  Aeronautics  and  Space  Administration, 
Ames  Research  Center,  NASA-TM-81 17 3 ,  1980. 

The  concept  of  a  cockpit  display  of  traffic  information  includes  the  integration  of 
air  traffic,  navigation,  and  other  pertinent  information  in  a  single  electronic 
display  in  the  cockpit.  Two  studies  were  conducted  to  develop  a  clear  and  concise 
display  format  for  use  in  later  full-mission  simulator  evaluations  of  the  CDTI 
concept.  Subjects  were  required  to  monitor  a  CDTI  for  specified  periods  of  time 
and  to  make  perceptual  judgments  concerning  the  future  position  of  a  single 
intruder  aircraft  in  relationship  to  their  own  aircraft.  Experimental  variables 
included  (1)  type  of  predictor  information  displayed  on  the  two  aircraft  symbols, 
(2)  time  to  encounter  point,  (3)  length  of  Lime  subjects  viewed  the  display, 
(4)  amount  of  practice,  and  (5)  type  of  encounter.  Results  show  that  (1)  length  of 
viewing  time  had  liule  or  no  effect  on  performance,  (2)  t  ime-t  o-enc  oun  t  er 
influenced  performance  with  ihe  straight  predictor  but  did  not  with  the  curved 
predictor,  (3)  and  learning  occurred  under  all  conditions. 

44,  O'Donnell  K.U.,  Ph.  D,  Secondary  Task  Assessment  of  Cognitive  Workload  in 
Alternative  •’uvkpit  Configurations.  Proceedings  of  the  AGARD  Conference  on  Higher 
Mental  functioning  in  Operational  Environments,  ACAKD-181,  October  1975. 

cTD-1  -  Cl 0-5.  ~ 

The  use  of  secondary  tasks  to  assess  primary  task  workload  is  not  a  new  concept, 
but  this  paper  discusses  this  concept  somewhat  differently  than  the  conventional 
methodology.  The  goal  was  to  permit  valid  workload  assessment  even  when  the 
subject  was  not  performing  up  to  "maximal"  levels.  Rather  Lhan  requiring  failure 
on  each  trial,  a  submaximal  workload  level  was  achieved.  Use  of  such  known  work¬ 
load  levels  in  the  laboratory  permits  the  experimenter  to  determine  the  meanings  of 
secondary  task  changes  with  respect  to  manipulations  of  the  primary  task  load. 
Once  the  meanings  are  established,  it  is  possible  to  use  the  secondary  task,  alone, 
to  evaluate  the  workload  of  unknown  primary  tasks.  The  itetn  recognition  technique, 
first  described  by  Sternberg  (1969),  is  used. 

Results  Lo  the  study  establish  the  feasibility  of  using  the  Sietnberg  set  in  the 
context  ot  a  complex  primary  task  which  is  essentially  psychomolor  in  nature.  The 
author  suggests  that  the  next  sLep  in  probing  for  sensitivity  and  validity  will 
require  its  use  with  a  primary  task  which  is  essentially  cognitive  in  nature. 
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45.  Ogden,  G.D.,  Levine,  J.M.,  and  Eisner,  E.J.,  Measurement  of  Workload  by 
Secondary  Tasks.  Human  Factors,  1979,  21,  529-548. 


The  purpose  of  the  article  was  to  review  the  post-1965  literature  on  the  use  of 
secondary  tasks  in  the  assessment  of  operator  workload.  Following  a  description  of 
the  underlying  theory,  the  literature  is  summarized. 

The  secondary  task  technique  is  used  to  determine  "how  much  additional  work  the 
operator  can  undertake  while  still  performing  the  primary  task  to  meet  system 
criteria."  The  secondary  task  technique  is  characterized  by  an  experimental 
situation  in  which  two  discrete  and  separate  tasks  are  performed  concurrently  with 
a  clear  emphasis  on  the  performance  of  one  of  the  tasks.  Use  of  the  secondary  task 
technique  involves  comparing  the  levels  of  performance  obtained  when  the  secondary 
or  loading  task  is  performed  alone  with  the  levels  obtained  on  the  same  task  when 
it  is  performed  with  the  primary  cask.  The  difference  between  the  performances 
obtained  under  the  two  conditions  is  then  taken  as  a  measure  or  index  of  the 
workload  imposed  by  the  primary  task.  The  assumption  is  that  the  presence  of  the 
secondary  task  does  not  interfere  with  the  performance  of  the  primary  task. 

The  secondary  task  technique  is  also  used  as  a  loading  task  to  assess  the  effects 
of  stress  on  operator  performance.  Operators  develop  strategies  of  responding  in 
order  to  contend  with  task  demands.  Typical  strategies  include  response  selection, 
response  omission,  response  queuing,  criteria  modification,  and  load  balancing. 
Secondary  task  selection  is  very  important  since  the  type  of  secondary  task 
employed  has  a  major  effect  on  the  response  strategies  available. 

A  classification  of  tasks  used  in  studies  measuring  workload  by  secondary  task 
techniques  was  included  in  the  discussion. 

Results  of  the  literature  review  indicate  that  research  on  the  use  of  secondary 
casks  in  the  assessment  of  operator  workload  has  not  yet  focused  upon  a  single  type 
of  secondary  task. 

46.  Palmer,  E.A.,  Jago,  S.J.,  Baty,  D.L.,  and  O'Conner,  S.L.,  Perception  of 
Horizontal  Aircraft  Separation  on  a  Cockpit  Display  of  Traffic  Information.  Human 
Factors.  1980,  22  (5),  605-620. 

The  experimental  tasks  described  in  this  article  required  teal  subjects  to  judge 
whether  an  intruder  aircraft  would  ultimately  pass  in  front  of  or  in  back  of  their 
own  aircraft  using  CDT1.  The  results  of  nine  experiments  are  described.  Results 
of  both  subjective  and  objective  data  reveal  that  pilot  performance  in  predicting 
horizontal  separation  between  their  own  aircraft  and  an  intruder  was  degraded  by: 

1.  Either  aircraft  turning  when  either  no  predictors  were  displayed  or  when 
the  predictors  did  not  include  turn-rate  information. 

2.  Increased  relative  heading  of  the  intruder  aircraft. 

Performance  is  improved  by: 

1.  Either  ground  referenced  or  own-ship  referenced  predictors  when  both 
aircraft  were  going  straight. 

2.  Including  turn-rate  information  or  both  the  ground  referenced  and 
own-ship  referenced  predictors  when  either  aircraft  was  turning. 
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Finally  performance  was  not  affected  by: 

1.  The  presence  or  absence  of  flight  path  history,  though  pilots  preferred 

having  history,  if  the  predictors  did  not  include  turn-rate  information. 

2.  The  presence  or  absence  of  ground  referenced  background,  although 
pilots  preferred  having  some  type  of  background. 

3.  The  rate  at  which  information  was  updated,  although  pilots  preferred 

having  their  own  position  updated  continuously. 

4.  The  speed  of  the  intruder  aircraft,  although  the  pilots'  response  bias 

did  not  change. 

Certain  limitations  in  research  methodology  are  cited  by  the  authors. 

47.  Parks,  0.,  Current  Workload  Methods  and  Emerging  Challenges.  In  Moray,  N. 
(Ed.)  Mental  Workload:  Its  Theory  and  Measurement.  New  York:  Plenum  Press,  1977, 
387-422. 

Predicting  and  measuring  methods  and  models  for  mental  workload  are  among  the  most 
urgent  needs  currently.  This  paper  is  concerned  with  the  needs  for  methods 
integration,  with  the  intent  to  convey  an  appreciation  for  the  display-control- 
human  factors  and  developmental  methods  used  in  design  efforts,  A  basic  framework 
is  presented  to  examine  applied  problems  as  a  baseline  for  correlating  basic  theory 
and  models.  The  discussion  offers  an  analytic  approach  used  for  identifying  system 
requirements  and  a  baseline  metric  for  evaluating  workload.  The  evolution  of  and 
rationale  for  procedures  used  to  identify  and  organize  system  requirements,  to 
identify  desired  crew  functions  and  to  select  equipment  is  described. 

The  use  of  analytic  and  workload  methods  is  discussed  in  context  with  the  system 
design  and  development  process.  The  paper  outlines  a  useful  approach  and 
identifies  representative  applied  questions,  trade-offs,  and  problems.  It  also 
describes  some  current  evaluations  of  electronic  displays  and  controls  to  reduce 
workload.  Finally,  the  concern  with  methrds  of  integrating  and  using  available 
techniques  to  meet  the  challenge  of  electronic  systems  is  discussed. 

48,  Pew,  R.W. ,  Secondary  Tasks  and  Workload  Measurement.  In  Moray,  N.  Mental 
Workload:  Its  Theory  and  Measurement.  New  fork:  Plenum  Press,  1977,  23-28. 

Secondary  task  methodology  derives  directly  from  the  limited  capacity 
single-channel  hypothesis.  To  the  extent  that  a  second  task  can  be  successfully 
performed  concurrently,  spare  capacity  is  available  over  and  above  that  required 
for  performance  of  the  primary  task. 

The  author  puts  forth  the  ideal  case  of  a  primary /secondary  task  ratio: 

%  workload  =  1  -  m-.-ntity  of  task  B  concurrently  with  Task  A 
quantity  of  Task  B  performed  alone 

Note :  Task  A  “  primary  task 

Task  B  =  secondary  task 
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The  reasons  why  this  ideal  has  never  been  achieved  in  practice  are  discussed. 
Methods  of  developing  measurement  scales  for  Task  B  are  also  discussed.  They  are 
(1)  using  scales  based  on  quality  of  Task  B  performance,  (2)  using  scales  based  on 
percentage  of  time  Task  B  is  performed,  and  (3)  using  several  versions  of  Task  B 
that  vary  in  demand  for  a  given  quality  level. 

The  author  concludes  that  as  long  as  no  better  theories  or  models  exist  to  predict 

workload  on  the  basis  of  a  priori  task  characteristics  or  to  clearly  identify  the 

dimensions  of  menLal  activites  that  contribute  to  workload,  the  best  alternative  is 
to  adopt  a  measurement  strategy  that  forces  the  subject  to  study  only  conditions 
for  which  the  single-channel  theory  can  be  used. 

49 .  Philipp,  U.,  Keiche,  U.,  and  Kirchner,  J.H.,  The  Use  of  Subjective  Rating. 
Ergonomi c s ,  1971,  14,  611-616. 

The  article  describes  an  evaluation  of  workload  in  air  traffic  control  tasks  in 
which  results  of  physiological  data  and  objective  work  factors  are  correlated  to 
the  operator's  subjective  assessment  of  workload.  A  scoring  method  and  procedure 
for  observer  rating  was  used  to  record  variations  in  the  subjective  feeling  of 

workload.  Results  of  subjective  ratings  have  been  related  to  different  measures  of 
information  processing.  In  subjective  ratings,  both  the  dimensions  "stress  of 

time"  and  "difficulty  of  the  control  task"  are  positively  correlated.  There  was  no 
significant  relation  at  the  95-percent  level  between  the  difficulty  as  sub¬ 
jectively  rated,  and  the  computed  difficulty.  In  all  cases,  results  of  objective 
measures  correlated  closely  with  subjective  measures.  Findings  indicate  that  for 
the  subjective  raLing  of  .sLress  in  a  man-machine  9ysLem,  the  method  of  observer 
rating  is  suitable. 

50.  Poock,  G.K.,  Using  PupillomeLry  as  an  Indicator  of  an  Operator's  Overload. 

Proceedings  of  Lhe  Human  Factors  Society  21st  Annual  Meeting,  October  17-20,  1977, 

197-198. 

Two  experiments  are  described  whose  results  indicate  lhai  complexity  and  mental 
overload  aspects  of  a  job  may  be  determined  by  analysis  of  pupillary  response. 

The  results  of  experiment  one  suggest  that  by  observing  a  group  of  people  doing  the 
same  job,  and  observing  where  Lhe  average  pupil  shows  a  severe  constriction  to 
below  a  normal  level,  one  might  be  able  to  identify  points  in  the  job  scenario 
where  the  operators  are  mentally  overloaded. 

Experiment  two  used  a  group  of  transpori/hel icopter/patrol  type  pilots  versus 
f igh t er/ati ack-Lype  pilots.  Each  subject  was  randomly  presented  a  series  of  10 
slides  showing  various  orientations  of  a  vertical  gyro  indicator  (VG1).  Results 
showed  no  significant  difference  in  the  two  groups  for  pupillary  size  changes. 
However,  latency  of  dilation  was  significant  at  Lhe  U.00U1  level  for  Lhe  slides. 
The  inverted  VG1  slides  caused  the  latency  of  dilation  to  increase  much  more  for 
Lhe  t ransport /hel i copter/ patrol  type  pilots  than  for  Lhe  f i gh ter/ a t tack-L y pe 
pilots. 
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51.  Price,  D.L.,  The  Effects  of  Certain  Gimbal  Orders  on  Target  Acquisition  and 
Workload.  Human  Factors,  1975,  17  (6),  571-576. 


The  effects  of  three  gimbal  order  on  target  detection,  recognition,  identification 
performance  and  operator  workload  were  evaluated  using  eighteen  experienced  pilots. 
"Gimbal  order"  refers  to  the  manner  in  which  a  sensor  is  suspended  beneath  an 
aircraft.  For  this  study,  a  2-axis  gimbal  was  used,  which  consisted  of  a  pair  of 
rings  pivoted  on  axes  aL  right  angles  to  each  other  so  that  one  is  free  to  swing 
within  the  other.  The  three  gimbal  orders  evaluated  were  roll-pitch,  yaw-pitch, 
and  pi tch-yaw . 

Results  indicate  that  gimbal  order  affecLed  target  detection  recognition  and 
identification  performance.  The  pitch-yaw  order  had  the  greatest  range-to-target 
scores,  and  it  also  had  the  lightest  workload. 

52.  Rasmussen,  J.,  Reflections  on  the  Concept  of  Operator  Workload.  In  Moray,  N. 
(Ed.)  Mental  Workload:  Its  Theory  and  Measurement.  New  York:  Plenum  Press,  1977, 
29-40. 

The  paper  discusses  concepts  of  workload  and  their  applications  to  the  human 
operator  in  an  automated  situation. 

Man  is  a  highly  adaptive  and  goa 1 -or i ent ed  information  processor.  Models  of  his 
performance  in  well-adapLed  situations  are  mainly  models  of  his  environment. 
Models  of  his  performance  must  also  reflect  the  limiting  properties  of  his  internal 
mechanisms.  The  author  makes  these  statements  in  response  to  the  current  tendency 
to  extrapolate  from  analytical,  quantitative  models  developed  for  manual  skills  and 
sensory-motor  responses  in  vehicle  control  and  apply  them  to  the  field  of  super¬ 
visory  control  disregarding  the  development  of  models  of  human  data  processing 
within  the  fields  of  artificial  intelligence,  robotics,  and  linguistic  research.  A 
new  philosophy  of  man-machine  systems  should  be  the  frame  of  reference  which  allows 
research  to  draw  upon  the  results  from  all  these  different  approaches  and  place 
emphasis  on  different  human  ability. 

53.  Rault,  A.,  Measurement  of  Pilot  Workload.  In  Moray,  N.  (Ed.)  Mental  Workload: 
Its  Theory  and  Measurement.  Nev  York:  Plenum  Press,  1977,  417-422. 

The  experimental  set  described  initially  uses  a  flighL  simulator  wi Lh  landing  Lasks 
and  various  levels  of  disturbance  and  ends  with  experiments  on  helicopters. 
Assessment  approaches  are:  psychological  measures  (tesis-Cooper  Harper  Scale), 
physiological  measurements  (cardiac  rhythm,  pulmonary  ventilation,  electromyography 
(emg)  of  the  neck  muscles,  and  e lec t ro-ocul gr aphy  (EOG)),  and  information  theory 
(the  transformation  type  of  approach  with  the  hypothesis  of  the  operator  being  a 
single  ch anne 1 ) . 

Conclusions  are  that  psychological  measures,  such  as  Cooper-Harper  scales  over  a 
homogeneous  population  of  10  test  pilots  gave  satisfactory  results.  Secondly, 
regarding  physiological  measures,  the  EOG  gave  Lhe  most  information.  The  position 
of  the  eyes  helped  to  clarify  pilot  behavior  and  establish  a  model.  Shortcomings 
of  physiological  measurements,  however,  are  the  great  intersubject  variations. 

The  information  theory  approach  failed  to  derive  valid  measurements  of  the  amount 
of  information  processed  by  the  pilot  because  some  of  the  hypotheses  for  the  use  of 
the  classical  tools  were  not  "alid  in  actual  situations;  i.e.,  linearity, 
s t  at i ona  r i t y . 
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54.  Rohmert,  W.,  Laurig,  W.,  Philipp,  U.,  and  Luczak,  H. ,  Heart  Rate  Variability 
and  Workload  Measurement.  Ergonomics ,  1973,  Volume  16,  No.  1,  33-44. 


Heart  rate  (HR)  as  a  more  or  less  suitable  indicator  of  strain  is  one  physiological 
variable  which  is  analyzed.  The  term,  heart  rate  variability,  implies  different 
kinds  of  variations  of  HR  which  become  portions  of  HR  variability.  These 
variations  depend  mainly  on  variations  of  stressors  and/or  variations  of  strain. 
This  study  focuses  on  3  parameters  used  to  describe  the  phenomenon  of  heart-rate 
variat ion. 

Those  parameters  used  were  (1)  amplitude  —  as  a  measure  of  amplitude  variations, 
the  difference  between  two  successive  heart  raLe  values;  12)  frequency  —  as  a 
parameter  to  be  attained  by  counting  the  number  of  changes  from  increasing  to 

decreasing  values  and  vice  versa,  and  (3)  variability  —  as  a  rate  using  a 
certain  number  of  heart  beats  as  a  base  (50). 

The  range  of  the  variation  of  the  above  parameters  is  discussed  using  both 
laboratory  and  field  investigation  results.  The  analyses  demonstrated  a  cor¬ 
relation  between  heart  rate  and  their  variability.  The  indication  seems  to  be  that 
when  heart  rate  variability  is  used  as  a  measure  of  strain  in  field  research, 

strain  might  be  under-assessed. 

55.  Rolfe,  J.M.,  Chappelow,  J.W.,  Evans,  R.L.,  Lindsay,  S.J.E.,  and 

Browning,  A.C.,  Evaluating  Measures  of  Workload  Using  a  Flight  Simulator. 

Proceedings  of  the  AGARD  Conference  on  Simulation  and  Study  of  High  Workload 
Operations .  AGARD-CP-146 ,  1974,  A4-1-A4-9. 

The  paper  describes  an  experiment  in  which  a  flighL  instrument  trainer  was  used  to 
evaluate  questionnaire,  performance,  and  activity  analysis  measures  of  pilot 

workload.  Attempts  were  made  to  distinguish  between  the  physical,  perceptual,  and 
mental  components  of  workload.  For  this  purpose,  three  flight  plans  were  devised 
of  approximately  equa 1  duration,  differing  markedly  wi  Lh  respect  to  the  three  above 
components.  bix  professional  pilots  flew  each  flight  plan  and,  afier  landing, 
completed  questionnaires  to  assess  the  workload  levels  and  the  task  content. 
During  the  flight,  video  recordings  were  made  of  the  pilot's  manual  and  communi¬ 
cation  activity.  Performance  during  Instrument  Landing  System  (1LS)  approaches 
immediately  before  and  after  the  experimental  flighLs  plans  was  also  measured. 

56.  Rolfe,  J.M.,  and  Lindsay,  S.J.E.,  Flight  Deck  Environment  and  Pilot  Workload: 
Biological  Measures  of  Workload.  Applied  Ergonomics,  4,  1973,  199-206. 

The  approach  in  this  study  on  workload  makes  three  assumptions; 

1.  All  tasks  impose  workload,  in  normal  as  well  as  abnormal  working 
8  1 1  uat i 008  . 

2.  Workload  varies  in  nature,  demanding  physical  and  menial  effort;  any 
measure  which  is  used  to  measure  workload  thoroughly  in  an  aircraft  cockpit  should 
be  sensitive  to  all  varieties  of  workload  which  will  occur. 

3.  Workload  varies  from  individual  to  individual. 


2b 


Based  on  the  above  assumptions,  the  study  of  workload  must  be  oriented  to  both  task 
and  operator.  It  may  seem  logical  to  place  performance  measures  at  the  head  of  any 
list  of  workload  measures.  However,  the  author  points  out  that  they  usually  reveal 
only  how  well  some  functional  system  criterion  is  met,  but  seldom  tell  the  price 
paid  in  operator  effort  in  meeting  this  criterion.  In  addition  to  measures  of 
performance,  which  indicate  how  well  the  task  is  performed,  it  is  necessary  to 

determine  how  much  it  costs  the  operator  to  achieve  his  level  of  performance.  In 

parallel  with  performance,  other  forms  of  measure  more  sensitive  to  the  assessment 
of  workload  should  be  considered,  such  as  observational  analysis  of  the  subject's 
performance  by  trained  observers,  subjective  assessments  obtained  from  the  crew, 
additional  performance  measures  using  loading  tasks  and  physiological  measures. 
All  of  these  techniques  should  be  reliable  and  should  not  impinge  on  the  operator 
with  the  effect  that  the  task  being  assessed  is  changed  by  the  presence  of  the 
assessment  measure;  methods  should  be  acceptable  to  the  subject;  and  results 
obtained  should  be  easy  to  interpret. 

Of  the  measures  assessed,  observation,  subjective  assessment,  and  physiological 
response  are  the  most  suitable  techniques  for  use  in  the  flight  environment.  Also, 
since  research,  to  date,  indicates  that  no  one  measure  can  adequately  indicate  "he 
nature  of  the  demands  the  Lask  imposes  or  the  efforL  expended  to  meet  the  demands, 

a  combination  of  measures  is  ttie  best  solution  to  the  problem. 

57.  Rolfe,  J.M.,  Whither  Workload?  Applied  Ergonomics,  March  1973,  8-10. 

One  approach  to  the  study  of  aircrew  workload  is  the  use  of  subjective  assessments. 
By  means  of  a  questionnaire,  aircrew  are  asked  to  indicate  the  extent  and  source  of 
their  workload  during  phases  of  flight.  A  study  was  conducted,  using  the  above 
subjective  technique,  to  compare  perceived  workload  between  single  seat  and  multi¬ 
crew  aircraft.  Results  reveal  the  extent  to  which  additional  crew  members  relieve 
the  pilot  of  ctrtain  loads  during  particular  phases  of  a  flight.  One  limitation  to 
the  questionnaire  method  of  obtaining  information  is  that  "it  gives  the  minimum  of 
information  relating  to  the  nature  of  the  load  imposed  on  the  subject  in  terms  of 
its  physical  and  mental  components."  Physiological  measurement  remains  nothing 
more  than  a  reliable  source  of  concomitant  material,  since  variations  in  activity 
can  be  brought  about  by  a  number  of  environmental  and  subjective  factors.  The  true 
meaning  of  physiological  changes  in  response  can  only  be  assessed  in  conjunction 
with  a  comprehensive  knowledge  of  the  task  situation. 

The  author  concludes  by  slating  that  in  terms  of  both  the  definition  of  workload 
and  Lhe  derivation  of  techniques  for  its  measurement,  the  inve9tigaLor  should 
consider  expanding  his  views  of  the  problem  to  accept  the  breadth  of  the  topic. 

58.  Roscoe,  A.H.,  Stress  and  Workload  in  Pilots.  Aviation,  Space,  and 

Environmental  Medicine,  1978,  630-636. 

Many  studies  have  highlighted  Lhe  increase  in  physiological  activity  which  occurs 
in  pilots  during  flight  and  especially  during  takeoff  and  landing.  As  an  example, 
it  has  been  clearly  shown  that  a  pilot's  heart  rate  increases  during  the  landing 
approach  to  reach  a  peak  at,  or  just  before,  touchdown,  investigators  have 
attributed  these  findings  to  the  effects  ot  workload  and  to  physiological  or 
emotional  stress.  It  is  important  to  be  able  to  differentiate  between  emotion- 
related  and  workload-related  aspects  of  flight.  In  this  paper,  the  author  assesses 
both  to  determine  their  relative  influence  on  heart  rate. 
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A  number  of  test  pilot' 8  heart  rate  responses  were  recorded  during  various  flight 
trials  involving  different  types  of  aircraft.  It  ia  concluded  that  heart  rate 
responses  in  experienced  pilots  are  influenced  almost  entirely  by  ’*orkload-related 
factors  and  not  by  emotional  stressors,  such  as  risk  and  anxiety. 

59.  Rosenbrock,  F.,  Hardware  Problems  in  Ergonomics  Measurements.  Ergonomics, 
1971,  14,  617-623. 

Assessment  of  the  operator's  workload 
problems  in  the  analysis  of  the  man 
measurement  is  further  complicated  by 
work  situation. 

This  paper  describes  the  analyses  of  an  actual  problem  (the  analyses  of  the  task 
and  workload  of  radar  controllers).  The  application  of  mul t ichan.iel  automatic 
data  acquisition  and  processing  is  utilized.  For  assessing  strain,  physiological 
variables  are  measured:  electrocardiogram  (ERG),  electro-oculogram  (EOG) ,  electro¬ 
myogram  (EMG)  of  a  back  muscle  and  respiration.  These  variables  are  correlated 
with  factors  of  stress  and  strain  using  a  coding  system  which  is  described.  The 
evaluation  of  a  multidimensional  work  process  study  is  synchronized  with  the 
physiological  data. 

60.  Rotondo,  Brig.  Gen.  G.,  Workload  and  Operational  Fatigue  in  Helicopter  Pilots. 
Proceeding  of  the  AGARD  Conference  on  Methods  to  Assess  Workload,  AGARD-CP-2 16 , 
April  1977,  Bl-l-Bl-1 1 . 

The  author  describes  and  analyzes,  in  greaL  detail,  the  nature  and  the  entity  of 
various  stressing  factors  that  constitute  the  physical  and  psychic  workload  of  the 
modern  helicopter  pilot.  Many  of  his  assessments  can  be  applied  to  the  broader 
category  of  pilots  in  general,  and,  in  fact,  the  author  makes  this  application.  He 
describes  various  components  of  fatigue  —  physical,  psychic,  and  emotive  — 
and  identifies  several  task-specific  causes  of  fatigue  for  the  pilot,  including 
noise,  vibration,  acceleration  and  variations  of  speed,  altitude,  pressure,  and 
temperature . 

He  concludes  that  flying  is  not  just  a  technical  or  operative  activity;  i.e.,  a 
job,  but  "a  vital  activity  and  an  'in  toto'  reaction  of  the  ego  to  the 
environment . " 

61.  Rouse,  W.B.,  Approaches  to  Mental  Workload.  In  N.  Moray  (Ed.)  Mental 

Workload:  Its  Theory  and  Measurement.  New  York:  Plenum  Press,  1977,  255-262. 

The  author  defines  workload  as  a  fraction  of  attention  and  intensity  of  effort. 
The  product  of  these  components  (i.e.,  fraction  of  attention  times  intensity  of 
effort)  offers  a  reasonable  composite  definition  of  workload.  Models  and  measures 
appropriate  to  these  definitions  are  considered,  including  information  theory 
models,  control  theory  models,  queuing  theory  models,  performance  measures, 
physiological  measures,  and  subjective  measures. 

In  concluding,  the  author  offers  a  synthesis  of  possible  approaches  to  the  problem 
of  mental  workload.  An  instantaneous  measure  of  workload  w(t)  might  be  given  by: 


is  one  of  the  moat  difficult  and  challenging 
-machine  interface.  The  problem  of  workload 
the  necessity  of  avoiding  interference  in  the 


w(l)  =  f(t)  x  i(t),  0<t<T 


where  f(t)  is  the  fraction  of  attention,  and  i(t)  is  the  intensity  of  effort,  at 
time  t,  and  (0,T)  is  the  time  interval  of  interest.  The  fraction  of  effort  f(t) 
might  be  predicted  using  a  queuing  model  while  the  intensity  of  effort  i(t)  might 
be  measured  physiologically  or  perhaps  assessed  via  subjective  measures. 

The  main  benefit  of  this  approach  is  that  it  is  not  limited  to  highly  structured 
tasks,  such  as  instrument  scanning  and  tracking. 

62.  Sanders,  A.F.,  Some  Remarks  on  Mental  Load.  In  Moray,  N.  (Ed.)  Mental 
Workload:  Its  Theory  and  Measurement.  New  York:  Plenum  Press,  1977,  41-77. 

This  paper  presents  an  analysis  of  the  concept  of  mental  load  from  the  human 
performance  theory  viewpoint. 

Following  a  discussion  of  the  theoretical  status  of  the  concept  of  mental  load,  an 
outline  is  given  of  the  main  procedures  of  measurement.  The  procedures  are  sub¬ 
divided  into  three  main  groups:  measures  of  behavior,  psychophysiology,  and 

subjective  judgment. 

The  next  section  describes  a  number  of  theoretical  bases  which  are  all  related  to 
the  limited  capacity  idea.  O' her  capacity  concepts  (in  terms  of  limited  capacity 
processors  and  effort)  are  described.  Three  types  of  processors,  with  varying 
applications,  are  outlined. 

Finally,  current  experimental  trends  are  discussed,  and  it  is  concluded  that  a 
multichannel  type  of  processor,  composed  of  a  network  of  internal  mechanisms,  is 
validated  by  the  data.  Consequently,  mental  load  cannot  be  conceived  of  as  a 

single  dimension.  Pending  applied  work  in  workload  measure  development,  a  task  may 
be  described  in  terms  of  a  pattern  of  mental  load. 

63.  Senders,  J.W.,  Axiomatic  Models  of  Workload.  In  N.  Moray  (Ed.)  Menta I 

Workload:  Its  Theory  and  Measurement.  New  Yoik.  Plenum  Tress,  1977,  263-207. 

The  author  puts  forth  several  mathematical  models  of  workload,  including  a  simple 
steady-state  model  ot  workload  which  includes  12  points.  He  summarizes  this  model 
by  stating  that  "workload  of  a  well-defined  task  is  the  length  of  a  vector  in  a 
space  defined  by  N  (number  of  actions)  elemental  type  workload  axes.  On  each  axis 
is  plotted  the  workload  corresponding  to  that  elemental  type.  Each  such  elemental 
workload  is  computed  by  taking  the  ratio  of  elemental  type  demand  to  elemental  type 
capacity . " 

The  other  models  included  are  simple  nonsteady-state  models  of  workload,  load 
component  generated  by  internal  switching,  load  component  generated  by  external 

switching,  switching  loads  and  time  cost,  and  motivation  and  effort.  Each  model  is 

clearly  delineated  on  a  point-by-point  basis  in  support  of  a  particular  hypothesis. 

64.  Sheridan,  T.B.,  and  Simpson,  R.W.,  Toward  the  Definition  and  Measurement  of 
the  Mental  Workload  of  Transport  Pilots.  Flight  Transportation  Lab/Man  Machine 
Lab,  Massachuse tt s  Institute  of  Technology,  Cambridge,  Mass.,  DOT-OS-70035,  1979. 

This  report  describes  research  at  the  M.l.T.  Flight  Transportation  Laboratory  on 
problems  of  defining  and  assessing  mental  workload.  The  primary  emphasis  is  on 
subjective  asse  ismeni .  One  result  is  the  generation  of  a  subjective  rating  method 
for  pilot  workload  (under  instrument  conditions)  modelled  on  the  Cooper-Harper 
r  at  i  up,  se  ale. 
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The  authors  point  to  the  changing  nature  of  pilot  workload,  describing  the  demands 
of  workload  as  a  "transient''  phenomena  when,  for  example,  a  routine  autopilot¬ 
monitoring  situation  suddenly  becomes  a  frantic  effort  to  control  an  unstable 
aircraft  or  avoid  midair  collision.  In  ascertaining  the  nature  of  mental  workload 
(versus  physical  workload  which  is  readily  measurable  by  physiological  indicators) 
the  authors  define  a  set  of  workload  factors,  one  for  each  component  of  their 
postulated  sensory-cogn i t i ve-raotor  system.  These  workload  values  are  a  function  of 
the  overall  task  demand  level  and  pilot  skill.  Thus,  the  authors  infer  that  by 
determining  workload  factors,  one  can  predict  when  overall  task  performance  is 
about  to  degrade. 

A  description  of  the  current  task  environment  and  task  characteristics  of  a 
transport  pilot  follows  the  theoretical  discussion,  along  with  a  summary  of 
objective  and  subjective  workload  measures. 

65.  Sheridan,  T.  ,  and  Strassen,  H.  ,  Definition,  Models,  and  Measures  of  Human 
Workload.  In  N.  Moray  (Ed.)  Mental  Workload:  Its  Theory  and  Measurement. 
New  York:  Plenum  Press,  1977,  219-233. 

Following  a  brief  review  of  workload  definitions,  empirical  measurements 
corresponding  to  those  definitions  are  discussed.  Measurement  techniques  include 
physiological  variables,  dual  or  secondary  task  methods,  information  measures,  task 
analyses,  attention  allocation,  and  subjective  measures. 

Some  metaphors  used  to  characterize  the  various  component  concepts  of  workload  are 
then  covered  including  stress  and  strain,  capacity,  and  attention  allocation.  The 
underlying  rationale  in  the  application  of  each  is  discussed. 

Finally,  the  formal  models  with  links  to  mental  load  are  discussed.  Those  models 
are  the  optimal  control  model,  decision  models,  information  channel  models,  and 
ac  t  i  va  t  i  on/uncc  r  t  a  i  n  t  y  models.  The  article  is  characterized  by  several  excellent 
illustrations  (see  figure  2  and  table  2). 
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66.  Spyker,  D.A.,  Stackhouse,  S.P.,  Khalafalla,  A.S.,  and  McLane,  R.C., 
Development  of  Techniques  for  Measuring  Pilot  Workload.  National  Aeronautics  and 
Space  Administration,  Contractor's  Report  NASA-CR-1888 ,  1971. 

"The  goal  of  this  study  was  to  provide  an  objective  method  of  assessing  information 
workload  based  on  physiological  measurements.  Information  workload  or  reserve 
capacity,  was  measured  using  a  visual  discrimination  secondary  task  and  subjective 
rating  of  task  difficulty.  The  primary  task  was  two  axis  (pitch  and  roll) 
tracking,  and  the  independent  variables  in  this  study  were  aircraft  pitch  dynamics 
“  t  ( K / S ,  K/S^ ,  and  K(S  +  l)  16  S/S^  ■  8S  =  16)  J  and  wind  gust  disturbances  (white 
noise  with  cut-offs  at  1.5,  2.5,  and  4.0  rad/sec.).  The  study  was  structured  to 
provide:  (1)  a  sensitive,  nonloading  measure  of  reserve  capacity,  and  (2)  an 
unencumbering  reliable  measure  of  the  psychophysiological  state.  From  these,  a 
measured  workload  index  (MW1)  and  physiological  workload  index  (PW1)  were 
extracted.  An  important  measure  of  the  success  of  this  study  was  the  degree  to 
which  the  MW1  and  PW1  agreed  across  the  243  randomly-presented,  4-minute  trials  (9 
subjects  X  9  tasks  X  3  replications)." 

The  somewhat  sophisticated  physiological  observations  which  were  considered  were: 

1.  E 1 ec t romyogram 

2.  Respiration 

3.  Vectorcardiogram  (VCG) 

4.  Skin  Impedance 

5.  Electroencephalogram 

6.  Eye  Mo 'emenL 


A  subjective  evaluation  of  task  difficulty  was  used  in  addition  to  the  above 
measures.  The  questionnaire  had  four  multiple  choice  questions  related  lo  the  dif¬ 
ficulty  of  the  primary  task  and  two  questions  relaLed  to  the  difficulty  of  the 
secondary  task.  Results  of  the  subjective  dai a  show  that  as  the  task  became  more 
difficult,  the  subjective  evaluation  of  difficulty  also  increased. 

67.  Stave,  A.M.,  The  Effects  of  Cockpit  Environment  on  Long-Term  Pilot 
Performance.  Human  Factors,  1977,  19  (5)  503-514. 


A  fixed-based  helicopter  simulator  was  used  to  examine  pilot  performance  under 
conditions  of  vibration,  noise,  and  fatigue.  Flight  periods  ranged  between  3  and  8 
hours,  using  test  subjects  with  a  minimum  of  1,000  hours  flying  time.  Results 
indicated  that  performance  seemed  to  depend  on  motivation  rather  than  fatigue; 
despite  the  onset  of  fatigue,  performance  not  only  did  not  degrade,  it  actually 
improved  with  time.  The  author  explains  this  phenomena  in  the  following  way:  as 
pilots  feel  the  onset  of  fatigue,  they  put  forth  increased  effort  to  compensate, 
which  results  in  an  initial  improvement  in  performance. 


Subjects  in 
performance . 
result  of  the 


the  study  suffered  from  "lapses"  resulting  in  abnormally  poor 
The  lapses  are  short  and  occur  at  unpredictable  times,  perhaps  the 
body's  reaction  to  fatigue  products  in  the  nervous  system.  They  are 
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also  attributed  to  a  reduction  in  the  amount  of  oxygen  in  the  air  breathed  by 
subjects  which  increased  the  number  of  blocks  and  prolonged  their  duration.  A 
momentary  lack  of  oxygen  in  the  brain  will  cause  a  block. 

If  it  can  be  shown  that  such  "lapses"  occur  in  actual  flight,  they  may  provide  a 
probable  explanation  for  many  "pilot  error"  accidents. 

68.  Steininger,  Dr.  P.,  Subjective  Ratings  of  Flying  Qualities  and  Pilot  Workload 
in  the  Operation  on  a  Short  Haul  Jet  Transport  Aircraft.  Proceedings  of  the  AGARD 
Conference  on  Studies  on  Pilot  Workload,  AGARD-CP-2 l 7 ,  1979,  B1Q-1-B10-7. 

The  article  describes  the  use  of  two  subjective  measures  used  by  subject  pilots  to 
evaluate  cockpit  layout  and  instrumentation.  The  pilots  were  employed  by  a  short 
haul  jet  transport  aircrafL  company.  The  measures  used  included  a  questionnaire  of 
82  fixed  items  answered  on  a  seven-step  rating  scale  and  a  structured  interviev 
covering  19  items.  The  author  addresses  the  pros  and  cons  of  subjective  measure¬ 
ment  as  cited  in  the  literature.  Likewise,  he  points  out  why  objective  measures 
may  not  always  be  so  valuable  as  an  assessment  measure. 

He  concludes  that  subjective  measures  are  valuable  instruments  provided  the 
following  criteria  are  met: 

1.  The  survey  should  cover  an  adequately  representative  sample  of  pilots. 

2.  The  state  of  training  and  experience  of  the  pilots  must  be  taken  into 
consideration  as  an  independent  variable  that  exerts  a  relevant  influence  on  the 
way  questions  are  answered. 

3.  The  questions  must  be  carefully  formulated  and  their  contents  must  be 
unambiguously  concerned  with  an  accurately  defined  element  of  the  system  or 
particular  criterion  to  be  judged. 

4.  The  set  of  questions  should  be  as  complete  as  possible;  i.e.,  must  cover 
not  only  all  the  individual  elements  of  the  system,  but  also  as  far  as  possible, 
the  dynamic  components  that  occur  during  the  real  operation  of  the  aircraft. 

5.  The  6et  of  questions  should  include  a  multiple  approach  in  order  to  cover 
more  than  one  view  of  the  problems  investigated.  That  means  the  application  of 
some  different  kinds  of  questioning  techniques  on  the  same  project;  i.e.,  rating 
techniques,  questionnaires,  checklists,  open  comments  on  prepared  questions,  and 
the  open  interview. 

69.  Strasser,  H.,  Physiological  Measures  f  Workload  Correlations  Between 

Physiological  Parameters  and  Operational  Performance.  Proceedings  of  the  AGARD 
Conference  on  Methods  to  Assess  Workload,  AGARD-CPP-2 16 ,  April  1977,  A8-1-A8-7. 

Stress  defined  as  input  load  of  man  at  work  may  be  assessed  by  means  of  operational 
measures  and  time  studies,  but  strain  as  the  individual  output  cannot  be  quantified 
without  considering  physiological  data.  Variability  of  some  physiological 
parameters,  mostly  used  in  the  field  of  workload  studies,  is  described;  and  ics 
practicality  is  discussed  for  application  in  field  studies.  Methodological 
problems  and  improvements  to  counteract  these  problems  are  also  discussed. 
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In  experimental  research  on  stress  and  strain,  very  often  correlations  between 
operational,  physiological,  and  subjective  rating  parameters  of  workload  are 
expected.  By  means  of  data  from  laboratory  studies  with  simultaneously  registered 
physiological  and  operational  performance  measures,  it  is  demonstrated  when 
correlations  can  be  found  and  when  they  cannot. 

The  influence  of  different  hypoxic  gas  mixtures  on  pursuit  tracking  and  on 
physiological  parameters  is  studied.  From  the  results,  several  conclusions  can  be 
made.  In  relatively  mild  hypoxia,  physiological  changes  are  present,  but  normally 
are  concealed  by  reactions  due  to  prolonged  test  time.  In  spite  of  statistically 
significant  physiological  effects,  no  noticeable  deteriorations  of  performance  in 
tracking  could  be  measured  in  hypoxia  down  to  an  hypoxic  gas  mixture  of  only  13 
percent  0^  in  inspired  air.  Not  until  before  11  percent  O2  significant  and 
meniionable  impairments  of  tracking  performance  were  found. 

The  same  is  true  of  noise.  Decreased  performance  in  noise  could  not  be  found,  but 
an  increased  level  of  heart  rate  indicated  the  stress. 

Results  indicate  that  physiological  indices  definitely  react,  even  in  low  workload 
situations,  in  order  to  bring  into  action  reserves  of  energy,  which  will  guarantee 
a  normal  performance.  Only  in  high  workload  situations  can  correlations  between 
performance  and  physiological  measures  be  expected. 

70.  Ursin,  H.  and  Ursin,  R.  ,  Physiological  Indicators  of  Mental  Workload.  In 
N  Moray  (Ed.)  Mental  Workload:  Its  Theory  and  Measurement.  New  York:  Plenum  Press, 
1977,  349-365. 

This  paper  discusses  some  available  physiological  measures  and  debates  whether  or 
not  they  are  applicable  to  measuring  mental  load.  The  main  premise  of  the  authors 
is  that  it  is  fairly  easy  to  measure  and,  therefore,  exclude  the  physical  workload 
and  measure  psychological  activation.  However,  serious  difficulties  exist  if  one 
tries  to  discriminate  between  emotional  factors  and  information  workload.  If  this 


?  I  discrimination  is  not  made,  measuring  "mental  workload”  does  not  involve  anything 

;  but  a  conventional  evaluation  of  activation. 

r  More  recently,  data  from  animal  and  human  experiments  are  helpful  in  furnishing  a 

I  1  more  refined  understanding  of  the  emotional  processes.  The  psychological  and 

|  physiological  processes  occurring  in  threatening  and  challenging  situations  show 

clear  dependency  on  experience  and  development  of  coping  behavior.  In  particular, 
it  is  important  to  know  at  which  stage  of  development  of  the  coping  process  the 
individual  is,  to  what  extent  he  evaluates  his  behavior  as  coping,  how  he  evaluates 
his  performance,  and  his  motivation  to  perform  in  that,  particular  task.  The 
physiological  responses  recorded  are  the  end  results  of  a  very  complex  psycho¬ 
logical  matrix.  There  is  some  evidence  of  specificity  in  the  endocrine  responses, 
but  these  have  not  been  established  in  any  detail. 

E 

71.  Verstynen,  H.A.,  Potential  Roles  for  the  Cockpit  Traffic  Display  in  the 
Evolving  ATC  System.  Society  of  Automative  Engineers  Technical  Paper  Series,  1980. 

!  In  this  paper,  the  author  begins  by  providing  a  few  simple  definitions  ot  CDH 

concept,  starting  with  the  cockpit  traffic  display  itself.  He  then  discusses  the 
i  multitude  of  potential  uses  for  CDT1,  with  special  reference  to  aircraft  appli- 

„  cations.  Some  potential  passive  roles  cited  for  CDTI  are  increased  situational 
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awareness,  blunder  detection  and  recovery,  reductions  of  separations,  hardware 
failure  detection,  and  recovery  and  automation  monitoring.  Active  roles  are 
pilot-controlled  spacing  and  merging,  condition  clearances,  pilot-controlled 
primary  separation,  landing  rollout,  turnoff,  and  taxiing  of  aircraft  in  low 
visibility  and  collision  avoidance. 

The  author  concludes  by  briefly  describing  che  research  efforts  of  both  the  NASA 
and  the  FAA  in  studying  the  advantages  and  disadvantages  of  CDT1,  in  a  realistic 
operational  environment,  for  a  broad  range  of  aircraft  classes  and  ATC 
envi ronments . 

72.  Wei  ford,  A.T. ,  Mental  Workload  as  a  Function  of  Demand,  Capacity,  Strategy, 
and  Skill.  Ergonomics ,  1978,  21,  151-167. 

The  author  outlines  a  conceptual  framework  in  order  to  view  the  many  problems  of 
defining  and  measuring  mental  workload.  Demand  and  capacity  are  the  areas  on  which 
the  paper  focuses.  Two  types  of  demand  are  distinguished;  the  amount  of  data  to 
be  handled  at  a  particular  moment,  and  the  amount  to  be  dealt  with  at  a  particular 
time.  Capacity  and  the  effects  of  various  factors  are  considered  in  relation  to 
concepts  of  a ignal-to-noi se  ratio  in  the  central  nervous  system.  Finally,  two 
approaches  to  the  measurement  of  workload  are  compared.  In  addition,  chronic 
stress  and  emotions,  often  overlooked  in  workload  analysis,  are  briefly  discussed 
as  long-term  factors  liable  to  affect  Workload. 

73.  Wickins,  C.,  Derrick,  W.,  Berringer,  D.  and  Micalizzi,  J.,  The  Structure 

of  Processing  Resources:  Implications  for  Task  Configuration  and  Workload. 
Proceedings  of  the  Human  Factors  Society  -  24th  Annual  Meeting,  1980,  253-256. 

With  respect  to  workload  assessment,  the  multidimensionality  of  processing 
resources  has  important  implications.  If  workload  is  defined  as  the  demands  of 
tasks  imposed  upon  the  operators'  limited  capacities,  then  adequate  workload 
measurement  must  delineate  and  account  for  the  various  dimensions  of  those 
capacities.  Evidence  abounds  for  a  multiple  resource  concept.  The  structure  of 
resources  is  defined  by  three  dimensions:  processing  stages  (perceptual  and 

central  processing  versus  response  selection  and  execution);  processing  modalities 
(visual  inputs  and  manual  responses  versus  auditory  inputs  and  vocal  response);  and 
processing  codes  (verbal  versus  spatial). 

The  series  of  experiments  described  in  the  paper  attempt  to  validace  the 
dimensionality  of  this  model,  as  part  of  a  broader  task  to  demonstrate  its  utility 
in  workload  measurement. 

Results  indicate  that  considerable  data  must  be  gathered  to  evaluate  the  utility  of 
the  model  in  accounting  for  task  interference  effects  and  workload  variance. 

74.  Wiener,  E.L.,  and  Curry,  R.  ,  Flight  Deck  Automation:  Promises  and  Problems. 
National  Aeronautics  and  Space  Administration,  Ames  Research  Center,  NASA- TM-81 206 , 
1980. 

The  paper  discusses  the  current  human  factors  concerns  in  flight-deck  automation 
and  offers  some  broad  design  guidelines.  In  addition,  some  automation-related 
aircraft  accidents  and  incidents  are  discussed  as  examples  of  automated  flight 
problems . 


37 


The  authors  voice  their  concerns  about  whether  many  flight-deck  functions  that  can 
be  automated,  should  be.  It  is  questionable  whether  total  system  safety  is  always 
enhanced  by  allocating  functions  to  devices  rather  than  hum*.n  operators,  and 
perhaps  flight  deck  automation  may  have  already  passed  the  point  of  optimality. 

75.  Wierwille,  W. ,  Williges,  C.,  Survey  and  Analysis  of  Operator  Workload 
Assessment  Techniques.  Patuxent  River,  Maryland,  Naval  Air  Test  Center,  Systems 
Engineering  Division,  Technical  Report  No.  S-/8-101,  1978. 

This  source  begins  with  a  description 
compile  an  exhaustive  bibliography  and 
workload,  specifically  aircrew  workload 
of  workload  assessment: 

1.  Subjective  opinion 

2.  Space  mental  capacity 

3.  Primary  task  assessment 

4.  Physiological  measures. 

These  four  categories  are  divided  into  28  specific  techniques,  shown  in  table  3. 
Each  of  these  techniques  is  described  in  detail,  along  with  an  equipment 
description  necessary  for  implementation,  pros  and  cons  of  the  technique,  and 
suggestions  for  future  research. 

The  report  also  acts  as  a  guide  in  selecting  a  workload  assessment  methodology  for 
aircrew  flight  test  and  evaluation  by  providing  specific  examples  of  research 
problems  and  determining,  through  the  use  of  rating  scales  and  checklists,  the  most 
applicable  workload  assessment  techniques. 

The  authors  conclude  that  surprisingly  little  data  are  available  on  the  development 
of  techniques  designed  specifically  for  flight  test  and  evaluation.  Although  this 
survey  points  to  several  approaches  that  are  potentially  useful,  no  single  tech¬ 
nique  can  be  recommended  as  the  definitive  measure  of  operator  workload.  The 
authors  suggest  the  use  of  multiple  measures  selected  from  the  four  major 
categories  described  in  the  report. 

Over  400  references  relating  to  operator  mental  workload  were  selected  and  reviewed 
for  this  report. 


of  the  in-depth  search  techniques  used  to 
review  of  literature  relevant  to  operator 
The  report  includes  four  major  categories 
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TABLE  3.  WORKLOAD  METHODOLOGIES  CLASSIFICATION 


Major  Categories 


1  Subjective 
Opinion 


Rating  Scales 


Interviews 


Jueat lonnaires 


II  Spare  Mental 
Capacity 


III  Primary 
Task 


Techniques  Within  Categories 


Task  Analytic 


Task  Component 

Time  Summation 
Information  - 
Theoretic 

Secondary  Task 
Nonadaptive, 

Ar  ithemet ic/ 
Logic 

Nonadaptive, 

Tracking 

Time  Estimation 

Adaptive , 
Arithmetic/ 
Logic 

Adaptive , 
Tracking 


Single  Measures 


Multiple  Measures 


Math  Model int 


IV  Physiological 
Measures 


Single  Measures 


FFF 

GSR 

EKG 

EMG 

EEG 

Eye  and  Eyelid 
Movement 

Pupillary 

Dilacion 

Miscle  Tension 

Heart  Rate 

Blood  Pressure 

Heart  Rate 
Variability 

Breathing 

Analysis 

Body  Fluid 
Analysis 

Handwriting 

Analysis 

Comb ined  Measures 


Speech  Pattern 
Analysis 
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76.  Wierwille,  W.,  Physiological  Measures  of  Aircrew  Mental  Workload.  Human 
Factors,  1979,  21,  575-553. 

Physiological  measures  of  workload  allow  a  global  definition  of  workload  in  which 
mental  workload  is  assumed  to  occur  when  any  conglomerate  of  behaviors  is  exhibited 
versus  the  assumption  that  the  human  operator  is  a  single-channel  sampling  device. 
In  this  paper,  the  class  of  workload  techniques  termed  "physiological"  can 
generally  be  discriminated  from  other  workload  techniques  by  the  fact  that  the 
changes  occurring  in  measures  are  largely  involuntary. 

Two  major  categories  were  formed,  single  measures  and  multiple  measures,  with  the 
first  of  these  further  subdivided  into  13  individual  measures.  Each  measure  is 
analyzed  individually,  even  though  data  on  more  than  one  measure  have  been  taken 
into  account  in  a  given  experiment.  The  objective  of  the  studies  described  was  to 
find  a  single  physiological  measure  that  reliably  reflects  change  in  operator 
mental  workload. 

The  results  indicate  that  few,  if  any,  of  the  physiological  measures  developed  for 
assessment  of  workload  are,  at  present,  proven  to  the  extent  that  they  can  be 
widely  applied  to  aircrew  mental  workload  problems.  Those,  however,  that  appear 
most  promising  are  pupil  dilation,  evoked  cortical  potentials,  and  body  fluid 
analysis.  In  any  case,  the  author  recommends  that  physiological  measures  should  be 
combined  with  behavioral  measures  to  obtain  an  adequate  description  of  workload  in 
an  aircrew  task. 

77.  Williges,  R.,  and  Wierwille,  W. ,  Behavioral  Measures  of  Aircrew  Mental 
Workload.  Human  Factors,  1979,  21,  549-574. 

The  purpose  of  this  article  is  to  provide  an  integrated  review  of  behavioral 
workload  measures  and  suggest  various  unresolved  research  issues  which  need  to  be 
addressed  before  implementation  in  the  operational  flight  test  and  evaluation 
environment.  Specifically,  14  methods  that  emphasize  behavioral  measures  are 
discussed . 

Of  the  14  general  approaches  reviewed,  no  single  technique  can  be  recommended 
as  the  definitive  behavioral  measure  of  operator  workload.  Because  of  the  multi¬ 
dimensionality  of  workload,  it  appears  unlikely  that  any  single  measure  will 
ever  suffice  completely.  The  strongest  research  support  exists  for  using  sub¬ 
jective  opinions  and  task  analytic  methods  involving  task  component/t ime  summation. 

Probably  the  most  reliaole  approach  is  through  the  use  of  multiple  measures 
including  dimensions  of  subjective  opinions,  spare  mental  capacity,  and  primary 
tasks,  as  well  as  physiological  correlates. 

78.  Young,  L.R.  and  Sheena,  D.,  Survey  of  Eye  Movement  Recording  Methods. 
Behavior  Research  Methods  and  Instrumentation,  1975,  Volume  7(5)  397-429. 

This  paper  describes  the  current  techniques  for  measuring  eye  movements,  including 
the  advantages  and  disadvantages  of  each.  The  physical  characteri sties  of  the  eye 
that  aie  used  in  eye  movement  measurement  are  covered.  In  addition,  major  eye 
movement  measurement  techniques  are  discussed  with  special  attention  given  to  new 
techniques.  Trade-offs  and  general  considerations  in  instrumentation  selection  in 
research  conclude  the  paper. 

Excellent  illustrations  and  a  summary  table  comparing  most  of  the  known 
eye-movemenL  measurement  techniques  characterize  the  paper. 


SUBJECT 

CDT 1 

Cockpit  Environment  -  General 
Physiological  Measures  -  General 

Heart  Rate  Variability 
Sinusarry thmi a 
Cardiac  Arrythmia 
Event-Related  Brain  Potential 
Oculometer  Measure,  Pupil lometry 
Workload  -  General 

Performance  Measurement 
Secondary  Tasks 
Tracking  Tasks 
Fatigue 

Objective  Workload  Measures 

Subjective  Rating 

Noise 
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TABLE  A-l.  DELINEATION  OF  FATIGUE  INDICATORS  (Sheet  1  of  4) 


Production  Rate  or 
Pr oduc  ti vi ly 


Irregularities  in 
timing  of  actions 

Disorganization  of 
skill  or  proficiency 


Reduction  of  speed  of 
task  performance 


Decrease  in  precision 
or  accuracy  of  per¬ 
formance 


Reference 


Chambers,  E.G.  Industrial  Fatigue. 
Occupational  Psychology,  Volume  35., 

1961,  44-57. 

Welford,  A.T.  Fundamentals  of  Skill, 

Methuen  and  Co. ,  London,  1968. 

Bartlett,  F.C.  Fatigue  Following  Highly 
Skilled  Work  Proc.  Roy.  Soc .  Ser.  B. 

Volume  131,  1943,  247-257. 

Singleton,  W.T.  Prologue  to  Seer  ion  1: 

Man.  Measurement  of  Man  at  Work, 

W.T.  Singleton,  J.G.  Fox,  and 
D.  Whitfield,  eds.,  Taylor  and  Francis, 
London,  1971,  15-16. 

Mohler,  S.R.  Fatigue  in  Aviation  Activities. 
Aerosp.  Med.  Volume  37,  1966,  722-732. 


Accuracy  or  quality 
of  secondary  task 
performance 


Brown,  l.D.  Measuring  the  Spare  Mental 
Capacity  of  Car  Drivers  by  a  Subsidiary 
Auditory  Task.  Ergonomics,  Volume  5, 
1962,  247-250. 
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Topic  Reference 

AFFECTIVE 

Self-ratings  of  Kashiwagi,  S.  Psychological  Rating  of  Human 

fatigue  feelings  Fatigue.  Methodology  in  Human  Fatigue 

Assessment  H.  Hashimoto,  K.  Kogi,  and 
E.  Grandjean,  eds.  Taylor  and  Fransis, 
London,  1971,  17-21. 

Schreuder,  O.B.  Medical  Aspects  of  Aircraft 
Pilot  Fatigue  with  Special  Reference  to  the 
Commercial  Jet  Pilot.  Aerosp.  Med., 

Special  Report  Pub.  37,  (4),  1966,  1-44. 

Luongo,  E.P.  Work  and  Physiology  in  Health 
and  Disease.  J.  Am.  Med.  Assoc., 

Volume  188,  1964,  27-32. 

Complaints  of  Bartley,  S.H.  The  Basis  of  Visual  Fatigue. 

Eye  strain  Am.  J.  Optometry  Arch.  Am.  Acad, 

or  discomfort  Optometry,  Monograph  30,  1947,  1-12. 

Direct  estimation 
of  fatigue  state 


Jenny,  L.L.,  Older,  H.J.  and  Cameron,  B.J. 
Measurement  of  Operator  Workload  in  an 
information  Processing  Task.  NASA 
CR-2150,  1972. 


Complaints  of 
lrritabil ity 
depression,  vague 
psychosomatic 
disorders,  etc. 

Feelings  of 
Impotence 


Mm# 


TABLE  A-l.  DELINEATION  OF  FATIGUE  INDICATORS  (Sheet  3  of  4) 


BEHAVIORAL 


Sensor-'  or  perceptual 
changjs 

Slowing  of  psycnomotor 
performance 

3imk.  value  or  ratio 


Wolford,  A.T.  Fundament al  of  Skill,  Methuen  ana 
Co . .  London ,  1968 . 


Fukui,  I.,  and  Morioka,  T.  The  Blink  Method  as  an 
Assessment  of  Fatigue  Methodology  in  Human 
Fatigue  Assessment,  K.  Hasnimoto,  K.  Hog; ,  and 
E.  Grandjean,  eds.,  Taylor  and  Francis,  London, 
1971,  23-30. 


Critical  fusi jn 
tr  equency 


Kogi  ,  K.  ant  Saito,  Y.  A  Factor  Analytic  Study  of 
Phase  Discrimination  in  Mental  Fatigue.  Method¬ 
ology  in  Human  Fatigue  Assessment,  <.  Hashimcto, 
K.  Kogi,  and  E.  Grandjean,  eds .  Taylor  and 
Francis,  London,  1 9 ? I  .  119-12?. 


Disintegrat  ion  of 
complex  action 
pattern* 


Handwriting  pressure, 
tapping  pressure 
and  speed  of 
finger  movement 


Tapping  rates 


Eye  Movements 


Finger  Tremor 


concent  rat  ion 
of  attention 


Blocking 


Wichdrewl,  decreased 
sexuality,  insomnia, 


Personal i cy  trai t a 
of  extroversion 


Manat,  H.  ar.d  Novak,  E.  A  Telemetnc  Method  for 
Assessing  Mental  Performance.  Methodology  in 
Human  Fatigue  Assessment,  K.  Haanimoto,  K.  Kogi, 
and  c.  Grandjean,  eds.,  Taylor  and  Francis, 

London,  1971,  129-135. 

Komoike,  Y.  and  Hongucni,  5.  Fatigue  Assessment  on 
Key  Punch  Operators,  Typists  and  Others.  Method¬ 
ology  in  Human  Fatigue  Assessment,  K.  Hashimoco, 

K.  Kogi,  and  E.  Grandjean,  eds.,  Taylor  and 
Francis,  London,  1971,  101-109. 

Grandjean,  E.  Introductory  Remarks . Metnodologv  m 
Human  Fatigue  As  se  sstnent .  K  .  Hashimoto,  K.  Kogi, 
and  E.  Grandjean,  eds.,  Taylor  and  Francis, 

London,  1971,  xvii-xxx. 

Ohtani,  A.  An  Analysis  ot  Eye  Movements  Turing  a 
Visual  Task.  Methodology  ir.  Human  Fatigue  Assess¬ 
ment,  K.  Hashimoto,  K.  Kogi,  ana  E.  Grandjean, 
eds.  Toylor  and  Francis,  london,  1971,  167-174. 

Nicnolson,  A.N.,  Hill,  L.E.,  Borland,  R.G.,  and 
Ferres,  H.M.  Activity  of  tne  Servou3  System 
During  the  Let-dovn,  Approach,  and  Landing:  A 
Study  of  Short  Duration  Hign  Workload.  Aerospace 
Med.,  volume  41,  1970,  436-446. 

Takakuwa,  E.  Maintaining  Concentration  as  a 
Measure  of  Mental  Stress.  Methodology  in 
Human  Fatigue  Assessment,  K .  Hashimoto,  K.  Kogi  , 
and  i.  Grandjean,  eds.  Taylor  and  Francis, 

London,  1971,  145-15U. 

Bills,  A.G.  3  locking:  A  New  Principle  of  Mental 
Fatigue.  Am  J.  Psychol.,  volume  a3,  1931, 

230-2-*  3 . 


Monlcr,  S.H. 
Aetuap.  Med 


Fatigue  in  Aviation  Activities. 
,  volume  37,  1966,  722-732. 


Eysencx,  H.J.  Tne  Biological  Basis  of  Personality. 
Charles  C.  Thomas.  Springfield,  111.  1967. 
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Topic 


Re  ference 


PSYCHOPHYSIOLOGICAL 


Excretion  of  urinary 
metabolites 


Circulatory  strain 


Lactic  acid  levels 
in  the  blood 


Rectal  temperature 


Muscular  tension 
or  contraction 


Percentage  of 
alpha  rhythms  in 
EEG 


Ocular  muscle 
st  rain 


Blood  condition 
and  neuromuscular 
exc itabil ity 

Disturbances  of 
visual  function 


Dukes-Dobos,  F.N.  Fatigue  from  the  Point  of  View 
of  Urinary  Metabolites.  Methodology  in  Human 
Fatigue  Assessment,  K.  Hashimoto,  K.  Kogi,  and 
E.  Grandjean,  eds.,  Taylor  and  Francis,  London, 
31-41. 

Donoso,  H.  ,  Apud ,  E. ,  and  Lungren,  N.P.  Direct 
Estimation  of  Circulatory  Fatigue  Using  a  Bicycle 
Ergometer.  Methodology  in  Human  Fatigue  Assess¬ 
ment,  K.  Hashimoto,  K.  Kogi,  and  E.  Grandjean, 
eds.  Taylor  and  Francis,  London,  1971,  53-60. 

Maitra,  S.R. ,  and  Koyal,  S.N.  Experimental  Studies 
of  Muscular  Fatigue  of  Bengalees  with  Increasing 
Work-loads  under  Different  Environmental  Condi¬ 
tions.  Methodology  in  Human  Fatigue  Assessment, 

K.  Hashimoto,  K,  Kogi,  and  E.  Grandjean,  eds., 
Taylor  and  Francis,  London,  1971,  71-84, 

Welch,  R.B.,  Longley,  E.O. ,  and  Lomaev,  0. 

The  Measurement  of  Fatigue  in  Hot  Working  Con¬ 
ditions  Methodology  in  Human  Fatigue  Assessment, 

K.  Hashimoto,  K.  Kogi,  and  E.  Grandjean,  eds., 
Taylor  and  Francis,  London,  1971,  85-90. 

Heminway,  A.  The  Physiological  Background  of 
Fatigue.  Symposium  on  Fatigue,  W.G.  Floyd  and 
A.T.  Welfork,  eds.,  H.K.  Lewis  and  Co.,  London, 

1953,  69-75. 

Kennedy,  J.L.  Some  Practical  Problems  of  the 
Alertness  Indicator.  Fatigue,  W.F.  Floyd  and 
A.T.  Welford,  eds.,  H.K.  Lewis  and  Co.,  London, 

1953,  149-153. 

Weston,  H.C.  Visual  Fatigue  with  Special  Reference 
to  Lighting.  Symposium  on  Fatigue,  W.F.  Floyd  and 
A.T.  Welford,  eds.,  H.K.  Lewis  and  Co.,  London, 
1953,  117-135. 

Juin,  G.  Protocol  and  Results  of  Medical  Inquiry  on 
the  Fatigue  of  Aircrews  Flying  on  Board  the  Boeing 
707.  Pilots  de  Ligne,  Paris,  December,  1961. 

Murphy,  M.R.,  and  Randle,  R.J.  Effects  of  Fatigue 
on  Visual  Accommodation  in  Air  Crew  Members.  Paper 
presented  at  the  42nd  annual  Meeting  of  the  Aero¬ 
space  Medical  Association,  Houston,  Texas,  1971. 
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Topic 

TASK  PERFORMANCE 


Record  of  actual 
crew  activities 
by  flight  segment 


Accuracy  of  quality 
primary  task 
performance 


Speed  and  task 
completion  time 


Average  duration 
of  task  elements 


Accuracy  or  quality 
of  secondary  task 
performance 


Probability  of 
target  acquisition 


Threshold  for 
detectable  error 


Completeness  and 
accuracy 


Reference 


Anon:  Some  Workload  and  Environmental  Character¬ 
istics  of  An  Air  Carrier  Short  Haul  Turbo-Jet  . 
Operation.  Summary  Report  on  a  UAL-ALPA  Joint 
Project  to  Evaluate  Pilot  Workload  on  B-737  Flight 
Operations,  United  Airline,  March  1969. 

Roscoe,  S.N.  Assessment  of  Pilotage  Error  in 
Airborne  Area  Navigation  Procedures.  Tech.  Rep. 
ARL-72-24/AFOSR-72-1 3 ,  Univ.  of  Illinois  Insti¬ 
tute  of  Aviation,  Savoy,  Illinois,  October  1972. 

Wianer,  A.  Electrophysiological  Measures  for  Tasks 
of  Low  Energy  Expenditure.  Measurement  of  Man  at 
Work,  W.T.  Singleton,  J.G.  Fox  and  D.  Whitfield, 
eds. ,  Taylor  and  Francis,  London,  1971,  61-74. 

Leplat,  J.,  and  Pailhous,  J.  The  Analysis  and 

Evaluation  of  Mental  Work.  Measurement  of  Man  at 
Work.  W.T.  Singleton,  J.G.  Fox,  and  D.  Whitfield, 
eds.,  Taylor  and  Francis,  London,  1971,  51-56. 

Rolfe,  J.M.  The  Secondary  Task  as  a  Measure  of 
Mental  Load.  Measurement  of  Man  at  Work.  W.T. 
Singleton,  J.G.  Fox  and  D.  Whitfield,  eds., 

Taylor  and  Francis,  London,  1971,  135-148. 

Zaitzeff,  L.P.  Aircrew  Task  Loading  in  the  Boeing 
Multimission  Simulator.  In  Proceedings  of  the 
56th  AGARD  Conference  on  Measurement  of  Aircrew 
Performance,  Brooks  AFB,  Texas,  1969,  8-8-3. 

Buckner,  D.N.,  and  McGrath,  J.J.  A  Comparison  of 
Performance  on  Single  and  Dual  Sensory  Mode 
Vigilance  Tasks.  Tech.  Rep.  8,  Human  Factors 
Research,  Inc.,  Los  Angeles,  1961. 

Jenney ,  L.L. ,  Older,  H.J.,  and  Cameron,  B.J. 
Measurement  of  Operator  Workload  in  an  Infor¬ 
mation  Processing  Task.  NASA  CR-2150,  1972. 
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Topic 

BEHAVIORAL 


Finger  tremor 


Critical  fusion 
frequency 


Eye  movements 
and  fixations 


Intonat ions 
of  speech 

Pilot  evaluation 
of  handling 
qualities 

Pilot  evaluation  of 
task  difficulty 


Cognitive,  perceptual, 
8ychomotor,  and 
sensory  task 
performance 


Reference 


Nicholson,  A.N.,  Hill,  L.E. ,  Borland,  R.G. ,  and 
Ferres,  H.H.  Activity  of  the  Nervous  System 
During  the  Let-down,  Approach  and  Landing:  A  Study 
of  Short  Duration  High  Workload.  Aerosp.  Med., 
volume  41,  1970,  436-446. 

Rey ,  P.  The  Interpretation  of  Changes  in  Critical 
Fusion  Frequency.  Measurement  of  Man  at  Work, 

W.T.  Singleton,  J.G.  Fox,  and  D.  Whitfield,  eds., 
Taylor  and  Francis,  London,  1971,  115-120. 

Shackel,  B.  Electro-oculography:  The  Electrical 
Recording  of  Eye  Position.  Proceedings  of  the 
Third  International  Conference  on  Medical 
Electronics,  1966,  322-335. 

Luk'yanov,  A.N. ,  and  Frolov,  M.V.  Signals  of  Human 
Operator  State.  NASA  TT  F-609,  1969. 

Cooper,  G.E.,  and  Harper,  R.P.,  Jr.  The  Use  of 
Pilot  Rating  in  the  Evaluation  of  Aircraft  Hand¬ 
ling  Qualities.  NASA  TN  D-5153,  1969. 

Krzanowski ,  W.J.,  and  Nicholson,  A.N.  Analysis  of 
Pilot  Assessment  of  Workload.  Aerosp.  Med., 
volume  43,  1972,  993-997. 

Jenney,  L.L. ,  Older,  H.J.,  and  Cameron,  B.J. 
Measurement  of  Operator  Workload  in  an  Infor¬ 
mation  Processing  Task.  NASA-CR-2150, 

1972  . 
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i 

Topic  Reference 

AFFECTIVE  j 

Self-ratings  Borg,  G.  Psychological  and  Phsyiological  Studies  ! 

of  perceived  of  Physical  Work.  Measurement  of  Man  at  Work.  I 

exertion  W.T.  Singleton,  J.G.  Fox,  and  D.  Whitfield,  eds., 

Taylor  and  Francis,  London,  1971,  121-128. 

( 

Operator  preferred  | 

levels  of  work 
intensity 

i 

i 

Direct  estimates  of  Jenney,  L.L.,  Older,  H.J.,  and  Cameron,  B.J.  I 

task  difficulty  Measurement  of  Operator  Workload  in  an  Infor-  • 

mation  Processing  Task,  NASA-CR-2 130 ,  1972.  j 
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DELINKAT ION  OK  WORKLOAD  INDICATORS  (Sheet  U  of  6) 


'1'opic 

TASK  REQUIREMENTS 
INPUT _ 

Task  requirements 
by  flight  segmenL 


Task  completion 
time  versus  time 
available 

fask-t ime 
stress  index 


Crew  task  demand 
e 1 eme  n  t  s 

Number  of  input 
channels  and 
signal  rate 

Amount  of  task 
i nformat ion 
available  and 
i ndex  of  qual i ty 


Reference 


Zipoy,  D.R.,  Preselaar,  S.J.,  Cargett,  R.E., 

Belyea,  I.L.,  and  Hall,  S.J.  Integrated  Infor¬ 
mation  Presentation  and  Control  System  Study. 

AF FDL-TR- 70-79 ,  volume  1,  Air  Force  Flight 
Dynamics  Lab,  1970. 

Price,  H.E.  ,  Hcnsberger,  W.D.,  and  Ereneta,  W.J.  A 
Study  of  Potential  Roles  of  Supersonic  Transport 
Crews  and  Some  Implications  for  the  Flight  Deck. 
NASA-CR-561,  1966. 

Klein,  T.J.  A  Workload  Simulation  Model  for 

Predicting  Human  Performance  Requirements  in  the 
Pilot-Aircraft  Environment.  Paper  presented  to 
the  Human  Factors  Society's  14th  Annual  Convention 
San  Francisco,  October  13  to  16,  1970. 

Gartner,  W.B.,  Ereneta,  W.J.,  and  Donohue,  V.R.  A 
Full  Mission  Simulation  Scenario  in  Support  of  SST 
Crew  Factors  Research.  NASA  CR-73096,  1967. 

Hartman,  B.O.  Time  and  Load  Factors  in  Astronaut 
Proficiency,  Symposium  on  Psychophysiological 
Aspects  of  Space  Flight,  B.E.  Flaherty,  ed., 
Columbia  Univ.  Press,  New  York,  19bl  ,  278-308. 

Leplat,  J.,  and  Pailhous,  J.  The  Analysis  and 

Evaluation  of  MenLal  Work.  Measurement  of  Man  at 
Work.  W.T.  Singleton,  J.G.  Fox,  and  D.  Whitfield, 
eds.,  Taylor  and  Francis,  London,  1971,  51-36. 
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Topic 

PSYCHOPHYSIOLOGICAL 
Heart  rate 


Elect  r  ical 
activity  of 
the  brain 


Muscle  activity  or 
tension  EMG 


Skin  resistance, 
GSR 


Blood  pressure 


Sinusarrythinia 


Evoked  cortical 
potent ial 


Urinary  excretion  of 
catecholamines,  meta¬ 
bolites,  electrolytes, 
and  simple  compounds 


Re  ference 


Corkindale,  K.G.,  Cutnming,  F.G.,  and  Hammerton- 
Fraser,  A.M.  Physiological  Assessment  of  Pilot 
Stress  During  Landing.  In  Proceedings  of  the  56th 
AGARD  Conference  on  Measurement  of  Aircrew 
Performance.  Brooks  AFB,  Texas,  1969,  9-9-4. 

Haider,  M.  Comparison  of  Objective  and  Subjective 
Methods  of  the  Measurement  of  Mental  Workload. 
Displays  and  Controls,  R.K.  Bernotat  and 
K.P.  Garnter,  eds.,  Swets  and  Zeitlinger, 
Amsterdam,  1972,  17-27. 

Corkindale,  J.G.,  Cutnming,  F.G.,  and  Hammerton- 
Fraser,  A.M.  Physiological  Assessment  of  Pilot 
Stress  During  Landing.  In  Proceedings  of  the  56th 
AGARD  Conference  on  Measurement  of  Aircrew 
Performance.  Brooks  AFB,  Texas,  1969,  9-9-4. 

Murrell,  K.F.  Temporal  Factors  in  Light  Work. 
Measurement  of  Man  at  Work,  W.T.  Singleton, 

J.G.  Fox,  and  D.  Whitfield,  eds.,  Taylor  and 
Francis,  London,  1971,  45-50. 

Ettema,  J.H.,  Zielhuis,  R.L.  Physiological 

Parameters  of  Mental  Load.  Methodology  in  Human 
Fatigue  Assessment,  K.  Hashimoto,  K.  Kogi ,  and 
E.  Grand  jean,  eds.,  Taylor  and  Francis,  London, 
1971,  137-144. 

Kalsbeek,  J.W.  Sinusarrythmia  and  the  Dual  Task 
Method  in  Measuring  Mental  Load.  Measurement  of 
Man  at  Work,  W.T.  Singleton,  J.G.  Fox  and 
D.  Whitfield,  eds,,  Taylor  and  Francis,  London, 
1971,  101-114. 

Wisner,  A,  ELectrophys iological  Measures  for  Tasks 
of  Low  Energy  Expenditure.  Measurement  of  Man  at 
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